# Common networking Protocol V2
This protocol defines a common way of communicating between server and client to setup an agreed upon environment for mods to register and use their own networking protocol components.
## Initial work
### Minecraft
Minecraft comes with a specification for custom payloads that can be sent within its protocols. This custom payload specification, as of version 1.20.4 of the game, looks as follows:
```plantuml
@startuml
interface "CustomPacketPayload" as spec {
void write(FriendlyByteBuf buf)
ResourceLocation id()
}
hide fields
@enduml
```
Where `FriendlyByteBuf` is Mojangs wrapper around the native Netty `ByteBuf` and `ResourceLocation` is Mojangs namespaced object identifier.
### Dinnerbones design of minecraft:register and minecraft:unregister
Originaly Dinnerbone worked on _Bukkit_ and with that he designed the initial protocol we still use to this day to determine what channel exist within a connection between a client and a server: https://web.archive.org/web/20220711204310/https://dinnerbone.com/blog/2012/01/13/minecraft-plugin-channels-messaging/
This original version has been overhauled a little, most noteably that the 16 character long strings are now `ResourceLocations` bit but the diagram for the `minecraft:register` and `minecraft:unregister` channels looks as follows:
```plantuml
@startuml
protocol "minecraft:register" as register
protocol "minecraft:unregister" as unregister
register : String channels
unregister : String channels
note as ChannelsFormattingNote
All channel ids that possibly could be send
on a connection by the sender of this payload
are concatenated and split with a "\0"
end note
ChannelsFormattingNote .. register::channels
ChannelsFormattingNote .. unregister::channels
hide methods
@enduml
```
A set of class models that could represent the entities for the payloads above would then be:
```plantuml
@startuml
class "MinecraftRegisterPayload" as register <<record>> extends CustomPacketPayload
class "MinecraftUnregisterPayload" as unregister <<record>> extends CustomPacketPayload
register : List<ResourceLocations> channels
unregister : List<ResourceLocation> channels
register : List<ResourceLocations> channels()
unregister : List<ResourceLocation> channels()
note as ChannelsFormattingNote
The write method implementation within these payloads
and the reader for these payloads would implement the
specification mentioned above. As such the representation
of these payloads across a connection would be a list of strings,
one for each of the channel ids in their fields respectively,
concatenated with a "\0"
end note
ChannelsFormattingNote .. register
ChannelsFormattingNote .. unregister
hide CustomPacketPayload Members
@enduml
```
Of note in this protocol is that the sending side communicates to the receiving side, which channels it can receive, not on which channels it will send payloads.
### Legacy Forge Networking implementation
Minecraft Forge and FML originally received an implementation based on the Dinnerbone protocol, but extended it by sending an additional set of payloads across the connection to determine eligibiltiy of the individual "channels".
:::info
Of note here is the concept of "channels" within this implementation. The platforms that use this system offered two kinds of implementations of these channels: Event driven where each payload type had its own channel. And Simple where per "Channel" a single name was used and the individual payload types where seperated by a single integer based descrimininator.
:::
The difference between Dinnerbones protocol and the legacy MCF implementation is the use of an unversioned side-channel negotiation using a set of payloads over the `fml:handshake` connection.
This side-channel followed the following protocol information:
```plantuml
@startuml
protocol "fml:handshake:99" as C2SAcknowledge <<C2SAcknowledge>>
protocol "fml:handshake:5" as S2CModData <<S2CModData>>
protocol "fml:handshake:1" as S2CModList <<S2CModList>>
protocol "fml:handshake:2" as C2SModListReply <<C2SModListReply>>
protocol "fml:handshake:3" as S2CRegistry <<S2CRegistry>>
protocol "fml:handshake:4" as S2CConfigData <<S2CConfigData>>
protocol "fml:handshake:6" as S2CChannelMismatchData <<S2CChannelMismatchData>>
S2CModData : Map<String, Pair<String, String>> mods
S2CModList : List<String> mods
S2CModList : Map<ResourceLocation, String> channels
S2CModList : List<ResourceLocation> registries
S2CModList : List<ResourceLocation> dataPackRegistries
C2SModListReply : List<String> mods
C2SModListReply : Map<ResourceLocation, String> channels
C2SModListReply : Map<ResourceLocation, String> registries
S2CRegistry : ResourceLocation registryName
S2CRegistry : bool hasSnapshot
S2CRegistry : byte[] snapshot
S2CConfigData : String fileName
S2CConfigData : byte[] fileData
S2CChannelMismatchData : Map<ResourceLocation, String> mismatchedChannelData
S2CModList <|--- C2SModListReply : Reply to
C2SModListReply <|--- S2CChannelMismatchData : Replied in case of failure
S2CRegistry <|--- C2SAcknowledge : Reply to
S2CConfigData <|--- C2SAcknowledge : Reply to
note as TriggeredByLogin
These payloads are send by entering the `NEGOTIATION` phase of the
minecraft networking protocol.
end note
TriggeredByLogin .. S2CModData
TriggeredByLogin .. S2CModList
TriggeredByLogin .. S2CRegistry
TriggeredByLogin .. S2CConfigData
hide methods
hide C2SAcknowledge members
@enduml
```
_Note: Allthough not displayed in the protocol above, the system on forge used SimpleChannel, meaning that the payload are always prefixed with their descriminator. The descriminator is shown above as the third section of the protocol element name_
Some details of the protocol are:
```plantuml
@startuml
protocol "fml:handshake:5" as S2CModData <<S2CModData>>
S2CModData : Map<String, Pair<String, String>> mods
note as ModData
The map for this part of the protocol has as keys the individual mod ids,
then for each mod id a pair, with as key the displayname,
and as value the version of the mod on the server.
end note
ModData .. S2CModData::mods
hide methods
@enduml
```
```plantuml
@startuml
protocol "fml:handshake:4" as S2CConfigData <<S2CConfigData>>
S2CConfigData : String fileName
S2CConfigData : byte[] fileData
note as ConfigFileName
The `fileName` part of the protocol is the exact path under
the "./config" directory
end note
note as ConfigFileData
The `fileData` part of the protocol is the exact binary contents
of the servers counter part.
end note
ConfigFileName .. S2CConfigData::fileName
ConfigFileData .. S2CConfigData::fileData
hide methods
@enduml
```
```plantuml
@startuml
protocol "fml:handshake:6" as S2CChannelMismatchData <<S2CChannelMismatchData>>
S2CChannelMismatchData : Map<ResourceLocation, String> mismatchedChannelData
note as NegotiationFailure
This payload is send when the server fails to find a common set of
network channels, mods, registries or datapack registries.
end note
NegotiationFailure .. S2CChannelMismatchData
hide methods
@enduml
```
Code wise the constructs for such an implementation of the protocol would look like:
```plantuml
@startuml
class C2SAcknowledge
class S2CModData
class S2CModList
class C2SModListReply
class S2CRegistry
class S2CConfigData
class S2CChannelMismatchData
S2CModData : Map<String, Pair<String, String>> mods
S2CModList : List<String> mods
S2CModList : Map<ResourceLocation, String> channels
S2CModList : List<ResourceLocation> registries
S2CModList : List<ResourceKey<Registry>> dataPackRegistries
C2SModListReply : List<String> mods
C2SModListReply : Map<ResourceLocation, String> channels
C2SModListReply : Map<ResourceLocation, String> registries
S2CRegistry : ResourceLocation registryName
S2CRegistry : bool hasSnapshot
S2CRegistry : byte[] snapshot
S2CConfigData : String fileName
S2CConfigData : byte[] fileData
S2CChannelMismatchData : Map<ResourceLocation, String> mismatchedChannelData
hide methods
hide C2SAcknowledge members
@enduml
```
This protocol is executed during the **login** phase of establishing of the connection.
Internally this is then executed using the following plan:
```plantuml
Server ->(20) Client : S2CModData
Server ->(20) Client : S2CModList
loop #LightBlue foreach : Registry
Server ->(20) Client : S2CRegistry
end
loop #LightGreen foreach : ServerConfig
Server ->(20) Client : S2CConfigFile
end
Client ->(20) Server : C2SModListReply
loop #LightBlue foreach : ReceivedRegistry
Client ->(20) Server : C2SAcknowledge
end
loop #LightGreen foreach : ReceivedServerConfig
Client ->(20) Server : C2SAcknowledge
end
```
### Fabric configuration phase protocol
With the introduction of the configuration phase, so did also come the need to refactor the way mods used and negotiated the available channels of custom packet payloads.
Modmuss50 from the Fabric team designed the first protocol in this PR: https://github.com/FabricMC/fabric/pull/3244
After attempts to implement this in NeoForge some drawbacks and missing features where discovered.
For completeness the protocol is described below:
```plantuml
@startuml
protocol "minecraft:register" as MinecraftRegister
protocol "minecraft:unregister" as MinecraftUnregister
protocol "c:register" as CommonRegister
protocol "c:version" as CommonVersion
protocol "ping" as Ping
protocol "pong" as Pong
MinecraftRegister : String channels
MinecraftUnregister : String channels
CommonVersion : Array<int> versions
CommonRegister : int version
CommonRegister : String phase
CommonRegister : Array<ResourceLocation> channels
note as ChannelsFormattingNote
All channel ids that possibly could be send
on a connection by the sender of this payload
are concatenated and split with a "\0"
end note
note as InitialTrigger
These payloads are send to the client by the server.
Starting with the "minecraft:register" payload, followed
by the "ping" packet
end note
ChannelsFormattingNote .. MinecraftRegister::channels
ChannelsFormattingNote .. MinecraftUnregister::channels
InitialTrigger .. MinecraftRegister
InitialTrigger .. Ping
Ping <|--- Pong: Reply to
MinecraftRegister <|--- MinecraftRegister : Reply with own channels
CommonVersion <|--- CommonVersion : Reply with accepted version
CommonRegister <|--- CommonRegister : Reply with own channel information
CommonRegister -[hidden]--> CommonVersion
hide methods
hide Ping members
hide Pong members
@enduml
```
The execution plan for this protocol is structured as follows:
```plantuml
@startuml
participant Server as Server
participant Client as Client
[-[#red]-> Client : Phase switch to "Configuration"
[-[#red]-> Server : Phase switch to "Configuration"
== Initialization ==
Server ->(20) Client : minecraft:register
note left
Channels are always optional and unversioned.
So all channels, known to the mod loader,
are listed in this initial payload.
end note
Server -[#LightBlue]>(20) Client : ping
note left
Additionally a Ping packet is sent.
The vanilla client will not respond to
"minecraft:register", but it will to
Ping packet. This allows for detection
of the type of client from the server side.
end note
Client ->(20) Server : minecraft:register
note right
The client now responds with the known channels.
As with the server, the mod loader lists all known
channels, unversioned and optional.
end note
Client -[#LightBlue]>(20) Server : Pong
note right
Because a ping packet was received we now also respond
with a pong. The response order is retained because
we use a serial sending logic and TCP connection types.
end note
== Client type determination ==
note across
Once the, up to, 4 packets are sent and their responses received
the server can determine what kind of client it is talking to.
This can then be used to selectively configure configuration tasks
for the next section.
Additionally the channel state is stored on the connection.
end note
== Configuration phases ==
loop foreach : ConfigurationTask
note across
Allthough vanilla implements a small amount of
configuration tasks it-self, most tasks come from mods.
As such this section will describe the tasks generically.
end note
Server ->(20) Client : DataPacket
note left
Most configuration tasks will send some form
of data gram packet to the client.
Examples are: Payloads to sync the registry or
server configuration entries, but also vanilla
uses this phase to sync the server resource
pack.
end note
Client -->(20) Server : AcknowledgementPacket
note right
The client can optionally acknowledge the
received packet.
If the server is expecting an acknowledgement
packet to be sent by the client it will wait
for that packet to be received by it, before
starting the next task.
end note
end
== Negotiation for Play ==
note across
We need to know what kind of protocol the server speaks
for the play phase.
end note
Server ->(20) Client : c:version
note left
Tell the client all versions of the
protocol that the server supports
end note
Client ->(20) Server : c:version
note right
The client answers with a single
selected version that it supports.
The logic for this is opaque to the
server, but it is suggested that the
client should pick the highest version
that they have in common, or disconnect
end note
Server ->(20) Client : c:register
note left
The server will send the payloads it
supports for the playphase of the protocol
end note
Client ->(20) Server : c:register
note right
The client will respond with the payloads it
supports for the playphase of the procotol
end note
Server --[#red]>] : "Phase switch to Play"
Client --[#red]>] : "Phase switch to Play"
@enduml
```
### Neoforge 20.4.70-beta implementation
During the initial networking migration within neoforged a new protocol was implemented.
It provided support for all features needed, but did not follow the "minecraft:register"
protocol as designed by dinnerbone:
```plantuml
protocol "minecraft:register" as MinecraftRegister <<ModdedNetworkQueryPayload>>
protocol "minecraft:network" as MinecraftNetwork <<ModdedNetworkPayload>>
protocol "neoforge:modded_network_setup_failed" as ModdedNetworkSetupFailedPayload <<ModdedNetworkSetupFailedPayload>>
protocol ModdedNetworkQueryComponent
protocol ModdedNetworkComponent
MinecraftRegister : Set<ModdedNetworkQueryComponent> configuration
MinecraftRegister : Set<ModdedNetworkQueryComponent> play
MinecraftNetwork : Set<ModdedNetworkComponent> configuration
MinecraftNetwork : Set<ModdedNetworkComponent> play
ModdedNetworkQueryComponent : ResourceLocation id
ModdedNetworkQueryComponent : Optional<String> version
ModdedNetworkQueryComponent : Optional<PacketFlow> flow
ModdedNetworkQueryComponent : boolean optional
ModdedNetworkComponent : ResourceLocation id
ModdedNetworkComponent : Optional<String> version
ModdedNetworkSetupFailedPayload : Map<ResourceLocation, Component> failureReasons
note as InitialTrigger
This payloads is send to the client by the server.
Starting with an empty "minecraft:register" payload,
followed by the "ping" packet
end note
note as ResultResponse
The server sends this payload to the client to
transmit the result of the payload version
negotiation
end note
note as FailureResponse
The server transmits this payload when the
negotiation fails.
This contains all the reasons in a display-
read form for each of the payload channels
end note
InitialTrigger .. MinecraftRegister
ResultResponse .. MinecraftNetwork
FailureResponse .. ModdedNetworkSetupFailedPayload
MinecraftRegister "0..*" -- "1..*" ModdedNetworkQueryComponent
MinecraftNetwork "0..*" -- "1..*" ModdedNetworkComponent
hide methods
```
If we look at the flow for this then the following execution plan can be defined:
```plantuml
@startuml
participant Server as Server
participant Client as Client
[-[#red]-> Client : Phase switch to "Configuration"
[-[#red]-> Server : Phase switch to "Configuration"
== Initialization ==
Server ->(20) Client : minecraft:register
note left
First an empty "minecraft:register" is transmitted.
This triggers the client to respond with its
payload channel information
end note
Server -[#LightBlue]>(20) Client : ping
note left
Additionally a Ping packet is sent.
The vanilla client will not respond to
"minecraft:register", but it will to
Ping packet. This allows for detection
of the type of client from the server side.
end note
Client ->(20) Server : minecraft:register
note right
The client now responds with the known channels.
end note
Client -[#LightBlue]>(20) Server : Pong
note right
Because a ping packet was received we now also respond
with a pong. The response order is retained because
we use a serial sending logic and TCP connection types.
end note
Server ->(20) Client : minecraft:network
note left
Once the negotiation completes the resulting network
structure is send over to the client to it can do
its own bookkeeping of which channels to use.
end note
== Client type determination ==
note across
Once the, up to, 5 packets are sent and their responses received
the server can determine what kind of client it is talking to.
This can then be used to selectively configure configuration tasks
for the next section.
Additionally the channel state is stored on the connection.
end note
== Configuration phases ==
loop foreach : ConfigurationTask
note across
Allthough vanilla implements a small amount of
configuration tasks it-self, most tasks come from mods.
As such this section will describe the tasks generically.
end note
Server ->(20) Client : DataPacket
note left
Most configuration tasks will send some form
of data gram packet to the client.
Examples are: Payloads to sync the registry or
server configuration entries, but also vanilla
uses this phase to sync the server resource
pack.
end note
Client -->(20) Server : AcknowledgementPacket
note right
The client can optionally acknowledge the
received packet.
If the server is expecting an acknowledgement
packet to be sent by the client it will wait
for that packet to be received by it, before
starting the next task.
end note
end
Server --[#red]>] : Phase switch to "Play"
Client --[#red]>] : Phase switch to "Play"
@enduml
```
## Analysis
With all of these network protocols described above, we can now
extract an opinionated list of advantages and disadvantages for
each protocol:
| Protocol | :thumbsup: Advantages | :thumbsdown: Disadvantages |
| -------- | -------- | -------- |
| Dinnerbone | <ul><li>Simple</li><li>Small</li><li>Understood by a large amount of platforms</li></ul> | <ul><li>Old</li><li>Unversioned</li><li>Not a lot of information</li><li>Normal payload flow driven by sender</li></ul> |
| Legacy Forge | <ul><li>Based on Dinnerbone's protocol</li><li>Side-Channel</li><li>Many features</li><li>Easy to implement</li><li>Fully registry synced</li></ul> | <ul><li>Propriatary protocol</li><li>All payload types required</li><li>No fallbacks</li><li>No versioning</li></ul> |
| Fabric | <ul><li>Build on the new configuration phase</li><li>Based on Dinnerbone's protocol</li><li>Partially versioned</li><li>Community driven</li><li>Side-Channel</li><li>Fully registry synced</li></ul> | <ul><li>Partially versioned</li><li>Not a lot of information</li><li>Not many features</li></ul> |
| NeoForge 20.4.70-Beta | <ul><li>Negotiation at the start</li><li>Simple</li><li>Server driven</li><li>Feature rich</li><li>Generic Packet Splitter</li><li>Partially registry synced</li></ul> | <ul><li>Breaks Dinnerbone's protocol</li><li>Unversioned</li><li>No fallbacks</li></ul>
## Requirements
From the analysis above we can collect a list of properties that we want the new protocol to have:
- Compatible with Dinnerbone's protocol
- Versioned
- With fallbacks, different protocols will always exist, we should do our best to be backwards, forwards and side-ways compatible
- Feature rich, but with optional sections
- Side-Channel driven
- Server driven, client interogation
- Supports generic packet splitting, at least on the receiver end. Sending is not needed. This is as such optional.
- Fully registry synced
- Payloads should be considered legitemate, so not limited to the 1Mb S2C and 32Kb C2S limits of unknown mojang payloads
- Optionally the mods list is synchronized between client and server, but it is not required.
## New protocol
### Concept
Based on the above requirements we can now start to design a new protocol.
The protocol will derive from the concept of channels:
```plantuml
@startuml
class Channel
Channel : ResourceLocation id()
Channel : Optional<String> version()
Channel : Optional<PacketFlow> flow()
Channel : boolean isRequired()
Channel : boolean isAdHoc()
Channel : boolean isConnected()
Channel : void connect()
Channel : void disconnect()
note left of Channel::id
This represents the id of the channel.
It has to be unique.
end note
note left of Channel::version
This represents the current version of the channel.
The version is optional
If the version is supplied then the other end
also needs the exact same version set.
end note
note left of Channel::flow
Defines the direction in which the payloads send
in this channel can be send. Payloads received
on an invalid side represent undefined behavbiour.
It is recommended that in such cases the connection
is aborted and an error is shown.
end note
note left of Channel::isRequired
Indicates whether this channel is required for
a proper connection to be established.
If at least one channel is found with this
property set, then no vanilla client can connect
to a modded server and vice versa.
end note
note right of Channel::isAdHoc
This flag is set when the channel is
created due to a `minecraft:register`
payload, however it was not negotiated
during the protocol initialization.
This can happen when connecting to
legacy systems, or plugin based
servers.
end note
note right of Channel::isConnected
Indicates whether the channel is connected.
For a required channel this is always true.
For an optional channel this depends on
whether the opposite end of the connection
also has a channel with the correct version
and flow set.
end note
note right of Channel::connect
This can be used to re-connect a channel
that previously had been connected.
Invoking this on an already connected channel has no effect.
end note
note right of Channel::disconnect
This can be used to disconnect a channel
that is currently connected.
Invoking this on an already disconnected channel has no effect.
Invoking this on a required channel, causes an exception.
end note
hide fields
@enduml
```
<br/>
<br/>
Within this concept of a channel, is the connection state important.
Because Dinnerbone's protocol only indicates whether the sender of the `minecraft:register` payload is able to receive payloads, and thus process them, on a given channel, we need to have the ability to disable the sending of specific payloads which are not supported by the receiving side.
Additionally an adhoc channel is something we need to keep in mind.
Because `minecraft:register` and its companion `minecraft:unregister` can be send at any time during the play phase, it is conceivable that a channel with an id previously unknown to us, would be send from the server. In this case an ad-hoc optional channel without version, or flow direction is immediatly created and can be used in the future for further sending **and** receiving of messages. Which will then directly be indicated by sending a `minecraft:register` response to the server with the new channel id as a listening channel included.
### Compatibility with legacy systems
To remain compatible, we will send all channels on which we know we can receive payloads (so with a flow bound to the current side of the protocol) in the initial `minecraft:register` (see below) payloads.
#### Protocol packets in legacy mode
As can be seen below, the protocol will remain in a "legacy"/initialization mode untill the protocol version and channels have been negotiated.
This means that for all intends and purposes the channels up untill that point are unversioned from the perspective of the protocol.
However they do have an inherit flow attached to them. Sending a protocol initialization payload on a channel in the wrong direction causes a disconnect for breach of protocol.
The protocol initialization channels are included in the initial `minecraft:register` payloads.
#### Fallback to legacy mode.
If the other side of the connection does not support the new protocol, then the system will abort the protocol flow after the `legacy packet exchange`, or during the `protocol version negotation` phases, depending on what is supported by the oposing side.
If the opposing side is a vanilla connection, then the system will determine channel compatibility and optionally, if required channels are found, disconnect the user with a message to install the relevant platform.
If the opposing side is a modded connection, but supports other protocols, like MCF, or older Fabric versions, then the system will switch into a `legacy protocol mode`. It will check for required channels, with versions or flow directions. If any such channel is present in the configuration of the network setup then the connection is disconnected with a relevant message to update the platform.
### Definition
#### Channel negotiation
```plantuml
@startuml
protocol ChannelSpecification
protocol ChannelConfiguration
enum PacketFlow {
CLIENT_TO_SERVER
SERVER_TO_CLIENT
BIDIRECTIONAL
NONE
}
protocol "c:supported_channels_negotiatie" as SupportedChannelsNegotiate
protocol "c:supported_channels_suggested" as SupportedChannelsSuggested
protocol "c:supported_channels_selected" as SupportedChannelsSelected
SupportedChannelsSuggested : ChannelSpecification[] configuration
SupportedChannelsSuggested : ChannelSpecification[] play
SupportedChannelsSelected : ChannelConfiguration[] configuration
SupportedChannelsSelected : ChannelConfiguration[] play
ChannelSpecification : ResourceLocation id
ChannelSpecification : Optional<String> version
ChannelSpecification : Optional<PacketFlow> flow
ChannelSpecification : boolean optional
ChannelConfiguration : ResourceLocation id
ChannelConfiguration : Optional<String> version
<> SuggestedDiamond
<> SelectedDiamond
SupportedChannelsSuggested -- SuggestedDiamond
SupportedChannelsSelected -- SelectedDiamond
SuggestedDiamond "1" *-- " * (play)" ChannelSpecification
SuggestedDiamond "1" *-- " * (configuration)" ChannelSpecification
SelectedDiamond "1" *-- " * (play)" ChannelConfiguration
SelectedDiamond "1" *-- " * (configuration)" ChannelConfiguration
ChannelSpecification "1" *-- "1?" PacketFlow
SupportedChannelsNegotiate <|--- SupportedChannelsSuggested : Client replies with
SupportedChannelsSuggested <|--- SupportedChannelsSelected : Server replies with
hide SupportedChannelsNegotiate fields
hide methods
note left of PacketFlow::NONE
If the direction NONE is chosen
then no payloads can be send on
this channel. It is then purely
used to ensure compatibility
between two parties
end note
@enduml
```
#### Protocol negotiation
```plantuml
@startuml
protocol "c:protocol_version_negotiate" as ProtocolVersionNegotiate
protocol "c:protocol_version_suggested" as ProtocolVersionSuggested
protocol "c:protocol_version_selected" as ProtocolVersionSelected
ProtocolVersionNegotiate <|--- ProtocolVersionSuggested : Client replies with
ProtocolVersionSuggested <|--- ProtocolVersionSelected : Server replies with
ProtocolVersionSuggested : int[] supportedVersions
ProtocolVersionSelected : int selectedVersion
hide ProtocolVersionNegotiate fields
hide methods
@enduml
```
#### Registry sync
```plantuml
@startuml
protocol "c:registry_id_sync_start" as RegistryIdSyncStart
protocol "c:registry_id_sync" as RegistryIdSync
protocol "c:registry_state_sync" as RegistryStateSync
protocol "c:registry_id_sync_end" as RegistryIdSyncEnd
protocol "c:registry_id_sync_acknowledged" as RegistryIdSyncAcknowledged
class RegistryNamespaceGroup << (M, orchid) >>
class RegistryNamespaceGroupBulk << (M, orchid) >>
class RegistryStateNamespaceGroup << (M, orchid) >>
class RegistryStateNamespaceGroupBulk << (M, orchid) >>
RegistryNamespaceGroup : String namespace
RegistryNamespaceGroup : RegistryNamespaceGroupBulk[] bulks
RegistryNamespaceGroupBulk : int bulkRawIdStartDiff
RegistryNamespaceGroupBulk : String[] names
RegistryStateNamespaceGroup : string namespace
RegistryStateNamespaceGroup : RegistryStateNamespaceGroupBulk[] bulks
RegistryStateNamespaceGroupBulk : int bulkRawIdStartDiff
RegistryStateNamespaceGroupBulk : byte flags
RegistryStateNamespaceGroupBulk : string|byte[] namesOrStates
note as RegistryBulkOffset
This field holds the offset of the id since
the end of the last bulk, regardless of namespace!
The first bulk has an offset compared to 0.
end note
note as FlagsNamesOrStates
The flags byte indicates with its least
significant bit if the array namesOrStates
contains pure names, and as such for those
entries the default state of that named
entry should be used, or if it contains
full states, serialized with the particular
registries state codec.
If names are serialized then the array
contains pure strings.
If full states are serialized using the
codec then it should be read as a set of
objects encoded to bytes using the codec
end note
RegistryBulkOffset .. RegistryNamespaceGroupBulk::bulkRawIdStartDiff
RegistryBulkOffset .. RegistryStateNamespaceGroupBulk::bulkRawIdStartDiff
FlagsNamesOrStates .. RegistryStateNamespaceGroupBulk::flags
FlagsNamesOrStates .. RegistryStateNamespaceGroupBulk::namesOrStates
RegistryIdSync "1" -- "*" RegistryNamespaceGroup
RegistryNamespaceGroup "1" -- "1+" RegistryNamespaceGroupBulk
RegistryStateSync "1" -- "*" RegistryStateNamespaceGroup
RegistryStateNamespaceGroup "1" -- "1+" RegistryStateNamespaceGroupBulk
RegistryIdSyncStart : Set<String> registries
RegistryIdSyncStart : Set<String> statefullRegistries
RegistryIdSync : String registryId
RegistryIdSync : RegistryNamespaceGroup[] namespaces
RegistryStateSync : String registryId
RegistryStateSync : RegistryStateNamespaceGroup[] namespaces
RegistryIdSyncEnd <|--- RegistryIdSyncAcknowledged : Client replies with
RegistryIdSyncStart -[hidden]--> RegistryIdSync
RegistryIdSyncStart -[hidden]--> RegistryIdSync
RegistryNamespaceGroupBulk -[hidden]--> RegistryStateSync
hide RegistryIdSyncEnd fields
hide RegistryIdSyncAcknowledged fields
hide methods
@enduml
```
#### Packet splitting
```plantuml
@startuml
protocol "o:split_packet" as SplitPacket
SplitPacket : byte stateFlags
SplitPacket : VarInt innerPacketId
SplitPacket : byte[] payloadSlice
note as PacketId
The inner packet id is always the same for the set of "o:split_packet" payloads
which make up the split packet.
end note
PacketId .. SplitPacket::innerPacketId
hide methods
@enduml
```
#### Legacy
```plantuml
@startuml
protocol "minecraft:register" as MinecraftRegister
protocol "minecraft:unregister" as MinecraftUnregister
MinecraftRegister : String knownPayloadIds
MinecraftUnregister : String forgottenPayloadIds
MinecraftRegister <|--- MinecraftRegister : When new channels, then receiver replies with the new ad-hoc channels
MinecraftUnregister <|--- MinecraftUnregister: When old channels: then receiver replies with old ad-hoc channels
MinecraftUnregister -[hidden]--> MinecraftRegister
note as ChannelsFormattingNote
All channel ids that possibly could be send
on a connection by the sender of this payload
are concatenated and split with a "\0"
end note
ChannelsFormattingNote .. MinecraftRegister
ChannelsFormattingNote .. MinecraftUnregister
hide methods
@enduml
```
#### Modlist negotiation
```plantuml
@startuml
protocol ModEntry << (M, orchid) >>
protocol "o:mod_list_client" as ModListClient
protocol "o:mod_list_server" as ModListServer
ModListClient : ModEntry[] modEntries
ModListServer : ModEntry[] modEntries
ModListClient : bool enabled
ModListServer : bool enabled
note as EnabledState
The user or server can disable this synchronization.
If it is disabled this will indicate as such.
end note
EnabledState .. ModListClient::enabled
EnabledState .. ModListServer::enabled
ModEntry : String modId
ModEntry : String version
ModListServer <|--- ModListClient : Client replies with
ModListServer "1" -- "1+" ModEntry
ModListClient "1" -- "1+" ModEntry
hide methods
@enduml
```
### Flow
#### Full protocol support
_Note by Tech: we can send c:protocol_version_negotiate, mc:register, Ping in that order and directly see from the client's first reply which kind of connection it is. Faster than 2 RTs._
```plantuml
@startuml
participant Server as Server
participant Client as Client
[-[#red]-> Client : Phase switch to "Configuration"
[-[#red]-> Server : Phase switch to "Configuration"
== Initialization ==
group Support legacy systems
Server ->(20) Client : minecraft:register
Server -[#LightBlue]>(20) Client : Ping
Client ->(20) Server : minecraft:register
Client -[#LightBlue]>(20) Server : Pong
end
note across
Once the, up to, 4 packets are sent and their responses received
the server can determine what kind of client it is talking to.
This can then be used to selectively configure configuration tasks
for the next section.
Additionally the channel state is stored on the connection.
end note
group Negotiate protocol version
Server ->(20) Client : c:protocol_version_negotiate
Server -[#LightBlue]>(20) Client : Ping
Client ->(20) Server : c:protocol_version_suggested
Client -[#LightBlue]>(20) Server : Pong
Server ->(20) Client : c:protocol_version_selected
end
note across
Once the, up to, 5 packets are sent and their responses received
the server can determine what the compatibility level of the client
it is talking to is.
This can then be used to selectively configure configuration tasks
for the next section.
If the compatibility with this protocol can not be guaranteed,
because the Pong packet was received before the version suggested
payload, then the system will switch into legacy mode.
end note
group Negotiate supported channels
Server ->(20) Client : c:supported_channels_negotiate
Client ->(20) Server : c:supported_channels_suggested
Server ->(20) Client : c:supported_channels_selected
end
== Before vanilla configuration payloads ==
note across
This needs to run before vanilla sends any packet,
especially the packet with tags, as it contains
int based ids for registry contents.
end note
group #Pink Perform registry sync
note across
This is an optional part of the protocol
end note
note across
This specific phase in the configuration performs a synchronization of the int based ids of registry entries.
This is always the first task to execute, which allows other task to use registry ids for their payloads
end note
note over Server
This process is triggered on the server side
by the game triggering the relevant configuration phase
end note
Server ->(20) Client : c:registry_id_sync_start
loop foreach : Registry
Server ->(20) Client : c:registry_id_sync
end
loop foreach : Statefull registry
Server ->(20) Client : c:registry_state_sync
end
Server ->(20) Client : c:registry_id_sync_end
note across
The server waits for an acknowledge ment, because the client
can and probably should process the registry ids on the main
thread, and the server should not continue with further tasks
untill it is guaranteed that the registry ids are processed
end note
Client ->(20) Server : c:registry_id_sync_acknowledged
end
group #lightgreen Optionally perform configuration sync
note across
This is an optional part of the protocol, it is examplary
shown here for neoforge.
Allthough the protocol does not specify how to synchronize
mod configuration, it hereby does specify when it should happen.
end note
note across
This is always the second task to execute, as it makes sure that
client has the same configuration as the server.
If this phase is skipped then defaults configurations are loaded and assumed
end note
Server ->(20) Client : neoforge:server_config_start
loop foreach : Config
Server ->(20) Client : neoforge:server_config_file
end
Server ->(20) Client : neoforge:server_config_end
note across
The server waits for an acknowledge ment, because the client
can and probably should process the configs on the main
thread, and the server should not continue with further tasks
untill it is guaranteed that the configs are processed
end note
Client ->(20) Server : neoforge:server_config_sync_acknowledged
end
== Configuration tasks ==
loop foreach : ConfigurationTask
note across
Allthough vanilla implements a small amount of
configuration tasks it-self, most tasks come from mods.
As such this section will describe the tasks generically.
end note
Server ->(20) Client : DataPacket
note left
Most configuration tasks will send some form
of data gram packet to the client.
Examples are: Payloads to sync the registry or
server configuration entries, but also vanilla
uses this phase to sync the server resource
pack.
end note
Client -->(20) Server : AcknowledgementPacket
note right
The client can optionally acknowledge the
received packet.
If the server is expecting an acknowledgement
packet to be sent by the client it will wait
for that packet to be received by it, before
starting the next task.
end note
end
group #lightblue Optionally perform mod list sync as configuration task
note across
This is an optional part of the protocol.
The moment were this synchronization happens does
not matter to the protocol.
We suggest it to happen as a configuration phase,
additionally this can be disabled by the user.
end note
Server ->(20) Client : o:mod_list_server
Client ->(20) Server : o:mod_list_client
end
== Hand-off ==
Server --[#red]>] : Phase switch to "Play"
Client --[#red]>] : Phase switch to "Play"
@enduml
```
#### Legacy protocol support
The flow diagram below assumes that for all `c:*` and obviously for the `o:*` and `neoforge:*` payloads it is first checked if the opposing side listens for that payload before excuting for example those configuration phases.
```plantuml
@startuml
participant Server as Server
participant Client as Client
[-[#red]-> Client : Phase switch to "Configuration"
[-[#red]-> Server : Phase switch to "Configuration"
== Initialization ==
group Support legacy systems
Server ->(20) Client : minecraft:register
Server -[#LightBlue]>(20) Client : Ping
Client ->(20) Server : minecraft:register
Client -[#LightBlue]>(20) Server : Pong
end
group Negotiate protocol version
Server ->(20) Client : c:protocol_version_negotiate
Server -[#LightBlue]>(20) Client : Ping
Client -[#LightBlue]>(20) Server : Pong
end
note across
The client did not reply with the suggested channels,
but the pong was returned. In such case we can assume
that the opposing side has no support for this protocol.
We enable legacy mode.
end note
== Before vanilla configuration payloads ==
note across
This needs to run before vanilla sends any packet,
especially the packet with tags, as it contains
int based ids for registry contents.
end note
group #Pink Perform registry sync
note across
This specific phase in the configuration performs a synchronization of the int based ids of registry entries.
This is always the first task to execute, which allows other task to use registry ids for their payloads
end note
note over Server
This process is triggered on the server side
by the game triggering the relevant configuration phase
end note
Server ->(20) Client : c:registry_id_sync_start
loop foreach : Registry
Server ->(20) Client : c:registry_id_sync
end
loop foreach : Statefull registry
Server ->(20) Client : c:registry_state_sync
end
Server ->(20) Client : c:registry_id_sync_end
note across
The server waits for an acknowledge ment, because the client
can and probably should process the registry ids on the main
thread, and the server should not continue with further tasks
untill it is guaranteed that the registry ids are processed
end note
Client ->(20) Server : c:registry_id_sync_acknowledged
end
group #lightgreen Optionally perform configuration sync
note across
This is an optional part of the protocol, it is examplary
shown here for neoforge.
Allthough the protocol does not specify how to synchronize
mod configuration, it hereby does specify when it should happen.
end note
note across
This is always the second task to execute, as it makes sure that
client has the same configuration as the server.
If this phase is skipped then defaults configurations are loaded and assumed
end note
Server ->(20) Client : neoforge:server_config_start
loop foreach : Config
Server ->(20) Client : neoforge:server_config_file
end
Server ->(20) Client : neoforge:server_config_end
note across
The server waits for an acknowledge ment, because the client
can and probably should process the configs on the main
thread, and the server should not continue with further tasks
untill it is guaranteed that the configs are processed
end note
Client ->(20) Server : neoforge:server_config_sync_acknowledged
end
== Configuration tasks ==
loop foreach : ConfigurationTask
note across
Allthough vanilla implements a small amount of
configuration tasks it-self, most tasks come from mods.
As such this section will describe the tasks generically.
end note
Server ->(20) Client : DataPacket
note left
Most configuration tasks will send some form
of data gram packet to the client.
Examples are: Payloads to sync the registry or
server configuration entries, but also vanilla
uses this phase to sync the server resource
pack.
end note
Client -->(20) Server : AcknowledgementPacket
note right
The client can optionally acknowledge the
received packet.
If the server is expecting an acknowledgement
packet to be sent by the client it will wait
for that packet to be received by it, before
starting the next task.
end note
end
group #lightblue Optionally perform mod list sync as configuration task
note across
This is an optional part of the protocol.
The moment were this synchronization happens does
not matter to the protocol.
We suggest it to happen as a configuration phase,
additionally this can be disabled by the user.
end note
Server ->(20) Client : o:mod_list_server
Client ->(20) Server : o:mod_list_client
end
== Hand-off ==
Server --[#red]>] : Phase switch to "Play"
Client --[#red]>] : Phase switch to "Play"
@enduml
```
#### Vanilla support
```plantuml
@startuml
participant Server as Server
participant Client as Client
[-[#red]-> Client : Phase switch to "Configuration"
[-[#red]-> Server : Phase switch to "Configuration"
== Initialization ==
group Support legacy systems
Server ->(20) Client : minecraft:register
Server -[#LightBlue]>(20) Client : Ping
Client -[#LightBlue]>(20) Server : Pong
end
note across
The client did not reply with the correct `minecraft:register`
payload, but the pong was returned. In such case we can assume
that the opposing side has no support for this protocol.
We enable vanilla mode.
end note
== Configuration tasks ==
loop foreach : ConfigurationTask
note across
Allthough vanilla implements a small amount of
configuration tasks it-self, most tasks come from mods.
As such this section will describe the tasks generically.
end note
Server ->(20) Client : DataPacket
note left
Most configuration tasks will send some form
of data gram packet to the client.
Examples are: Payloads to sync the registry or
server configuration entries, but also vanilla
uses this phase to sync the server resource
pack.
end note
Client -->(20) Server : AcknowledgementPacket
note right
The client can optionally acknowledge the
received packet.
If the server is expecting an acknowledgement
packet to be sent by the client it will wait
for that packet to be received by it, before
starting the next task.
end note
end
== Hand-off ==
Server --[#red]>] : Phase switch to "Play"
Client --[#red]>] : Phase switch to "Play"
@enduml
```
### Packet splitting flow
#### Encode
##### Small enough
```plantuml
@startuml
participant Sender as Sender
participant PacketSplitter as Splitter
participant Encoder as Encoder
Sender -> Splitter : Attempts Send
activate Sender
activate Splitter
Splitter -> Splitter : Write packet
Splitter -> Splitter : Check size
note across
This causes an additional call to the underlying packet
end note
Splitter -> Encoder : Continue processing original packet
Encoder --[#red]>] : Send packet out over network
deactivate Splitter
deactivate Sender
@enduml
```
##### Too big
```plantuml
@startuml
participant Sender as Sender
participant PacketSplitter as Splitter
participant Encoder as Encoder
Sender -> Splitter : Attempts Send
activate Sender
activate Splitter
Splitter -> Splitter : Write packet
Splitter -> Splitter : Check size
note across
Packet is determined to be larger then 8Mb (as of writing)
end note
Splitter -> Splitter : Split data
loop foreach : Payload slice
Splitter -> Sender : Send wrapped slice
activate Sender
Sender --> Splitter : Passthrough of split packets
activate Splitter
Splitter --> Encoder : Immediatly encode split packets
deactivate Splitter
deactivate Sender
end
Encoder --[#red]>] : Send packet out over network
deactivate Splitter
deactivate Sender
@enduml
```
#### Decode
##### Small enough
```plantuml
@startuml
participant Decoder as Decoder
participant PacketSplitter as Splitter
participant Receiver as Receiver
[--[#red]> Decoder : Receive packet from the network
activate Decoder
Decoder -> Receiver : Decode Packet and handoff for processing
note across
The PacketSplitter is completely skipped here.
It is only triggered if a split packet payload is received.
end note
Receiver -> Receiver : Check thread and/or process
deactivate Decoder
@enduml
```
##### Too big
```plantuml
@startuml
participant Decoder as Decoder
participant PacketSplitter as Splitter
participant Receiver as Receiver
loop foreach : Split packet slices
[--[#red]> Decoder : Receive split packet payload from the network
activate Decoder
Decoder -> Splitter : Hand-off to packet splitter as its receiver
Splitter -> Splitter : Store payload slice and inner payload id
deactivate Decoder
end
note over Splitter
Once the final split packet slice is received
the processing continues.
end note
Splitter -> Splitter : Combine all payload slices
activate Splitter
Splitter -> Decoder : Decode payload
activate Decoder
note across
It should be impossible for this decoded packet
to be a packet with a split payload as its contents.
end note
Decoder --> Receiver : Trigger receiver for decoded packet
deactivate Splitter
deactivate Decoder
@enduml
```
Sign in with Wallet
Connect another wallet