Try   HackMD

Super Ultra Galaxy Omega Infinity Illusion game modding guide for beginners pre-alpha.

THIS GUIDE IS UNDER CONSTRUCTION! Some sections still only have basic outlines or stubs. Please help flesh these sections out if you can! You can edit this file by entering edit mode at top left.

NOTE: This guide is written for games based on mono. Newer games by Illusion (aka ILLGAMES) that use IL2CPP have many differences in the way plugins are written, and therefore this guide will probably not be as helpful.

TODOs

  • making and importing canvas guis from unityeditor, then using them in-game
  • making plugins and hooks that play well with others and are resiliant to game updates
  • include things from mod-modelling pins
  • include things from mod-programming pins

Who is this for?

This guide is for everyone that wants to make or modify plugins for Illusion games that use the BepInEx modding framework.

Prerequisites

  • The game to mod with BepInEx installed
  • Something you personally want to make - a feature idea or a bug to fix.
  • Will to RTFM and keep trying and failing until you git gud.
  • Basic modding/programming knowledge is very helpful but not necessary since you can learn as you go.

What games does this apply to?

  • Recent Illusion games made in Unity (Koikatsu or newer)
  • To a lesser degree all other games made in Unity (5.0 or newer)

Common classifications of mods

Plugins / assembly edits

Unity is using C# as its scripting language. In most cases all of the game code is inside a folder name Managed in dll files called assemblies. This code is responsible for all logic in the game and can be modified to modify or add features.

  • Assembly file edits - Permanent changes to the dll files that need to be redone on every game update.
  • Plugins - Custom code loaded by a plugin loader that don't make any permanent changes to the game code, so they are likely to still work after game updates.

Assets / zipmods / content mods

Unity compiles assets into either resource files (placed inside the *_Data folder by the compiler), AssetBundles (can be placed anywhere and created later), or Addressables (they are built on top of AssetBundles so they're stored in the same way).

All of these are proprietary formats and are not easily readable. They include various pieces of content used by Unity applications such as 3D models, animations, sounds, textures, scenes, materials and shaders.
It is important to note that AssetBundles do not contain code. Instead, AssetBundles contain Monoscripts. When a MonoBehaviour is configured, certain fields will serialize down to the disk when saved. This same data will be written to an AssetBundle when built.

New content (e.g. clothes, accessories) is usually added in AssetBundles that are loaded by game code at runtime (when the game is running). It is possible to modify this data and effect a change in the application's behaviour. For example, if a model is replaced in an AssetBundle, the application will load the modified model instead of the original.

Hardmods vs Softmods

Modifying Assemblies, AssetBundles, etc. in-place by replacing or otherwise changing their contents is known as Hardmodding.
The limitations of hardmodding are often enough to discourage the practice, however it remains a quick and useful approach to determine whether or not modding is possible for an application.
Drawbacks include:

  • Loss of the original data. When data is modified in-place, the original information is overwritten. In this way it usually becomes a choice of one or the other. Instead of the introduction of new content, the original content is replaced. It is also necessary to keep a backup to remove the modification.
  • Large file sizes. AssetBundles can be very large depending on how the application was designed. If a single sound clip is replaced in a bundle containing hundreds, it's quite impractical to provide this data for others.
  • Legal issues. Distributing entirely original content is fine, but distributing files that were authored by another raises concerns of piracy. For a hardmod to be entirely original content, all assets in a particular bundle would need to be replaced, which can be impractical or impossible depending on how tightly it integrates with the rest of the application.
  • Workflow. It's possible to publish Assetbundles from the Unity editor, however no tooling is provided to modify existing AssetBundles in-place. 3rd party software exists to extract or modify them but each has its own set of benefits and drawbacks:
    • UABEA
    • AssetStudio
    • uTinyRipper
    • SBUGS

Such drawbacks can be mitigated through Softmodding. In order for the application to read data from an AssetBundle it must first load data from the disk. This often happens in code which can be modified by plugins with Harmony - an alternative to modifying assembly files directly. One such implementation of this is Sideloader.
The high level goal of Sideloader is to provide a means of intercepting AssetBundle load requests, redirecting them or otherwise extending the list of loaded bundles such that no hardmods are necessary.
While BepInEx is an universal plugin loader that works on nearly all Unity games, there is no catch-all solution for resources that can be applied generically across all Unity applications. Just as each application is unique so too is how it reads and presents its data to the user.
Observing a non-destructive approach to modding across both code and content will allow modders to provide users with a convenient installation or uninstallation experience.

Modding communities

Basics

Things you should at least skim through.

Advanced

Guides that assume you have some specific goal in mind.

Plugins / Game code modifications

This section will focus on making changes to how the game operates.

Check the introduction above for a brief explanation of what a plugin and an assembly is, and a list of related guides.

What can you actually do?

It is possible to change content or behaviour in a Unity application or introduce new content or behaviours. If you can achieve something with code in a personal Unity project, a plugin can do it at runtime in an existing title.
Plugins can achieve an extra layer of functionality through the libraries included with BepInEx. With Harmony, Monomod and Cecil it becomes possible to modify or hook code with each library providing its own benefits and limitations.

Some examples of things that can be done:

  • Translating games which did not ship with a particular language.
  • Adding Multiplayer or VR support to titles which did not ship with it.
  • Porting content between games including models, animations and maps even if they were not created in Unity.
  • Integrating a game with hardware such as adding gamepad support.

It's difficult to capture the broad scale of what plugins can do as the possibilities grow with each Unity version. Something that is very difficult to achieve in Unity 5 may be a lot simpler in Unity 2019. To this end, it is important to study the Unity documentation for the Unity version that the application shipped with.

Which games can be modded?

Basically all games that use Unity game engine can be patched in one way or another. BepInEx supports everything starting from Unity 4.0 (although full support starts from 5.0). Games using Unity older than 4.0 can be patched with a legacy plugin framework like IPA or simply have their assemblies edited directly.

More modern Unity versions support IL2CPP code compilation mode (as opposed to the existing Mono/C# mode). It basically translates C# into C++ and compiles it as such. Because of this, it's much more difficult to edit the code directly (there are no assembly files to edit and all metadata is stripped). BepInEx, starting at version 6.0, supports patching games compiled as IL2CPP, but there are still some major downsides - it's much more difficult to read game code and plugins made for IL2CPP games are mostly incompatible with plugins made for Mono games and vice versa.

Skills required

  • Basic knowledge of programming in C# or experience in a different, similar programming language (Java, Python, C, JS, etc.) is very useful, but not absolutely required. In fact, many people have learned programming by modding these games.
  • Looking stuff up on Google
  • Stealing code from StackOverflow answers

Don't be afraid to ask for help in a related modding community, but please always at least try to solve the issue by yourself - this is the best way to learn and people will absolutely know if you didn't even try.

The skills you learn here might seem fairly useless outside of modding, but that is definitely not the case. You will learn how the relevant systems work on low levels and skills that might bring your skill up to 11 (for example how C# is compiled into IL and how IL can be written directly).

Software required

In order to efficiently write plugins an IDE or C# aware editor is strongly recommended.
At this point in time the best software to achieve this would be one of:

Being able to write code is only part of the solution. A great deal of time may be spent reverse engineering the application. For example if your goal is to add a new "Item", it becomes necessary to study how the application builds up its list of items, how it presents them in the world and how you can construct a new item that will not cause instability in the software.
Various tools exist to perform reverse engineering for .NET Assemblies:

  • dnSpy - Modified version of ilSpy made specifically for modding. Has debugging and code editing support (the dnSpyEx fork is latest).
  • ilSpy - Arguably the best decompiler, it tends to produce the cleanest code and is fully open source. dnSpy is using an older version of ilSpy, so if it fails to decompile something correctly, try using ilSpy instead.
  • dotPeek - Pretty good, but it's mostly geared towards testing normal applications instead of modding. Included in Resharper.
  • Reflector - It's useful for developers to see how their code gets compiled, but doesn't offer much for modding.

With decompilation and analysis software, it becomes possible to study the code that a Unity application is executing. This is Static Analysis, as the code is not running at the time of inspection.
Dynamic Analysis is possible where memory can be inspected at the time of execution, allowing you to see the values of fields.

getting and installing

bepinex install and enabling console

github git and hosting code

more in section # git

example plugins and stuff

Reading and understanding game code

  • dnspy search and analyze
  • how to string search the source dnspy export to project
  • dnspy decompilation configuration and why to use
    • config for diffing 2 game versions
    • understanding coroutines and lambdas
    • clicking on member doesn't show it, turn on compiler generated stuff
  • runtime editor and dnspy integration
  • connections between code, in-game scenes, and assets and how to discover them
  • find constants and strings
  • find in-game asset in assetbundles
  • find asset loaded from code
  • IL disassembly

Running custom code in a game

Modifying Assembly-CSharp.dll

hard modding and its limitations
–- editing in dnspy
–- common issues, missing refs, bad decompilation results

Using a plugin framework

why bepinex, legacy support
– history and different frameworks, whats supported
– how bepinex works
folder structure
runtime flow chart
plugins vs patchers

Creating a BepInEx plugin

– whats needed
– step by step in VS make a solution and project
- versions of .net csproj nuget etc
- how to tell what net to use 35 vs 46 and why, core and il2cpp, what happens if you pick wrong version / typeload exceptions in 35 runtime on 46 ass, works other way
- nuget packages vs direct refs
- old new csproj and package format, why old unless experienced
- make something simple like output to log when pressing a button

Harmony / HarmonyX

what is it and why

Warning: This guide assumes the use of HarmonyX (a fork of Harmony modified for better performance, less necessary code boilerplate, and better error handling/reporting). Some parts of this guide will not work in original Harmony. HarmonyX is backwards compatible with Harmony and most of the original documentation still applies.

how it works

Harmony Method Parameters

class __instance: Instance of class being used.
var parameter: Access to parameter used by the method. Can have the ref modifier attached ref var parameter to
var ___fieldname: Access to class fields useful when not publically accessable via instance.

Class based Patching

This is the original format patches were written in Harmony. The entire assembly is scanned for classes with HarmonyPatch attributes and specially named methods inside of them.

Patched via Harmony.CreateAndPatchAll(Assembly.GetExecutingAssembly()) or harmonyinstance.PatchAll()

[HarmonyPatch(typeof(class), nameof(class.method))]
internal class customclassname
{
    internal static void Prefix(){};
    internal static void Postfix(){};
}

Here is an example used by the MoreAccessories Plugin. The class being patched is the HSprite class which handles most of the GUI in a game scene.

[HarmonyPatch(typeof(HSprite), nameof(HSprite.AccessoryProc))]
internal static class HSpriteAccessoryProc_patch
{
    internal static void Prefix(HSprite __instance, HSceneSpriteCategory ___categoryAccessory){}
}

Anotation Based Patching

This method of patching was added in HarmonyX and is the recommended default. It allows for faster patching (no need to scan the whole assembly) and less boilerplate code. Some of the new HarmonyX functionality is only available in this patching style.

Patched via Harmony.CreateAndPatchAll(typeof(Hooksclass)) or harmonyinstance.PatchAll(typeof(Hooksclass))
Not patched via Harmony.CreateAndPatchAll(Assembly.GetExecutingAssembly()) or harmonyinstance.PatchAll()

instance class Hooksclass
{
    [HarmonyPrefix]
    [HarmonyPatch(typeof(class), nameof(class.method))]
    internal static void method(){}
}

Advanced Patching: Transpilers and Compiler Generated Classes

Unfortunatly at times you might want to patch via a transpiler when you want to execute or replace a portion of code in a method. Using this form of patching requires some understanding of assembly code to patch. This is a method that the MoreAcessory overhaul uses.

Here are a couple of examples that MoreAccessories deals with. The following lines of codes are examples of code that limits Accessories from exceeding 20. Fortunatly this example can be patched normally using the above methods just using the Transpiler method, but there is a caveat, compiled classes.

public void examplemethod()
{
    if(!RangeEqualOn(0,slot,19)) return;
    if(slot>=20) return;
    for(int i = 0; i < 20; i++) {}
}

When dealing with compiled classes, the code can become game dependent as it is possible the the compiled class isn't shared or has changed between games. Unfortunatly Annotation based patching doesn't seem to be compatible when working with compiled classes.

Compiled classes tend to be generated when working with UniTasks, Coroutines, Anonomous methods and other ways.

Here is an example of a UniTask/Coroutine tasks being patched to replace an RangeEqualOn Check. Here the Main class being patched is ChaControl Class and the method is the ChangeAccessoryAsync method's generated class.

//internal static IEnumerable<MethodBase> TargetMethods()//for multiple methods to be patched
internal static MethodBase TargetMethod()
{
    MethodBase methodbase;
#if KKS
    methodbase = AccessTools.Method(AccessTools.TypeByName("ChaControl+<ChangeAccessoryAsync>d__483, Assembly-CSharp"), "MoveNext");
#elif KK || EC
    methodbase = AccessTools.Method(AccessTools.TypeByName("ChaControl+<ChangeAccessoryAsync>c__Iterator12, Assembly-CSharp"), "MoveNext");
#endif
    return methodbase;
}

internal static IEnumerable<CodeInstruction> Transpiler(IEnumerable<CodeInstruction> instructions)
{
    var instructionsList = instructions.ToList();
    //return yield each instruction as desired with inserts (or skip instructions to exclude) where desired.
}

Tips: AccessTools.TypeByName

  • Namespaces are seperated by '.'
  • Classes are seperated by '+'
  • When in major doubt, try printing typeof(class).AssemblyQualifiedName
    An disgusting KK/EC Example:AccessTools.TypeByName("ChaCustom.CustomAcsChangeSlot+<Start>c__AnonStorey0+<Start>c__AnonStorey1, Assembly-CSharp")

Further reading

Debugging Harmony patches

Because of being dynamically generate Harmony patches can be difficult to debug. There are a couple of ways of doing it - you can add extra logging, debug with dnSpy, or increase Harmony logging level.

Manual extra logging

Adding extra logging is the easiest, but usually also the least efficient since you often have to do a few recompiles to narrow down on the issue. It's not possible to add extra logging to result of transpliers easily.

dnSpy debugging

You can drop dnSpy breakpoints in your prefix and postfix patches and they will work as expected. Patched methods are replaced, therefore dropping a breakpoint in the target method will not work - the breakpoint is added to the original copy of the method, not our new patched version, so it will never trigger. Patched methods appear as ??? in dnSpy stack trace window.

To debug patched methods themselves in dnSpy, start by setting HarmonyBackend=cecil in BepInEx.cfg - this will use cecil to generate patches which will allow dnSpy to see them (normally dynamic methods are generated for the patches, but switching to cecil generates individual new assemblies with the patches). You still can't drop a breakpoint in the original method, but if you drop a breakpoint in a method that is called by the patched method, you can then see the patched method in the stack trace window and jump inside it. You can also step into the patched method.
Warning: Setting the backend to cecil can on some systems make connecting dnSpy debugger take forever (if you try to start the game from dnSpy then it will work, but harmony will take very long to apply patches so it will take forever anyways).

Built-in Harmony logging

Finally you can crank up the harmony logging level to show much more information. The IL log level is especially useful when dealing with transpliers, but it will create very large amounts of log spam so it's best to set it right before your patch runs, and turn it off after it finishes. You can set the log levels by changing HarmonyLib.Tools.Logger.ChannelFilter.

In recent HarmonyX versions you can add the HarmonyDebug attribute to your patches. Doing so will automatically enable all debug logs for a particular patch without having to modify log levels.

Patch assembly dumping / Debugging transpliers

Finally, when debugging Transpliers you can make Harmony dump the patched method into a separate dll file so that you can open it in dnSpy and inspect the resulting IL code (very useful when you are getting patching crashes).
























try
{
    // Tell HarmonyX to start dumping any patches it generates to mmdump directory in your game dir
    Environment.SetEnvironmentVariable("MONOMOD_DMD_DUMP", "./mmdump");
    // You must use the cecil backend to dump patch assemblies
    Environment.SetEnvironmentVariable("MONOMOD_DMD_TYPE", "cecil");
    
    // Remove old dumps so it's easier to find the latest dumps
    if (Directory.Exists("./mmdump"))
    {
        foreach (FileInfo file in new DirectoryInfo("./mmdump").GetFiles())
            file.Delete();
    }
    
    // Run your patches that you want to dump here
    Harmony.CreateAndPatchAll(Assembly.GetExecutingAssembly(), PluginInfo.PLUGIN_ID);
}
finally
{
    // Turn dumping off so you aren't flooded with dll files
    // Need to do this in finally in case the patching process crashes (this wouldn't be called then)
    Environment.SetEnvironmentVariable("MONOMOD_DMD_DUMP", "");
}

KKAPI and other API plugins, why use them and how

Application Programming Interface (API) plugins function as a intermediate between the game and other plugins by providing access to commonly used methods and can provide a common structure to use.

KKAPI provides several common structures such as the CharaCustomFunctionController (CCFC) which is attached to all characters in a game. The CCFC is useful for when workng on plugins that work with specific characters. The CCFC provides methods that are called at the required point in time such as when saving or loading the character.

In order to use KKAPI, it's best practice to install the nuget package for the game being modded rather than directly referencing a dll.

The following example of the basic layout for CCFC for most plugins using a rough example of extracting and saving Extended Save Data using the previous standard handling data

internal Awake()//typically in BaseUnityPlugin class provided by BepInEx
{
    KKAPI.Chara.CharacterApi.RegisterExtraBehaviour<CCFC_Class>(ExtendedSaveKey);//ExtendedSaveKey is typically the GUID for the original version
}
public class CCFC_Class : KKAPI.Chara.CharaCustomFunctionController
{
    private Dictionary<int, Example_Class> ExampleCoordFriendlyMethod;
    private Example_Class Now_Coord_Data;
    protected override void OnReload(GameMode currentGameMode, bool MaintainState)
    {
        var plugindata = GetExtendedData();
        if (plugindata == null) return;
        // You can only store basic types in the data dictionary (int, bool, string, etc.) and their arrays
        // This example saves a complex data structure (Dictionary) by first serializing it to a byte array
        if (plugindata.data.TryGetValue("Example", out var ByteData))
            // MessagePackSerializer should be in either Assembly-CSharp-firstpass.dll or MessagePack.dll, depending on the game
            Now_Coord_Data = MessagePackSerializer.Deserialize<Dictionary<int, Example_Class>>((byte[])ByteData);
    }

    protected override void OnCardBeingSaved(GameMode currentGameMode)
    {
        var plugindata = new PluginData();
        plugindata.data.Add("Example", MessagePackSerializer.Serialize(ExampleCoordFriendlyMethod));
        SetExtendedData(plugindata);//KKAPI Method that abstracts ExtendedSave using ExtendedSaveKey
    }
    
    protected override void OnCoordinateBeingSaved(ChaFileCoordinate coordinate)
    {
        var plugindata = new PluginData();
        plugindata.data.Add("Example", MessagePackSerializer.Serialize(Example_Class));
SetCoordinateExtendedData(plugindata);
    }

    protected override void OnCoordinateBeingLoaded(ChaFileCoordinate coordinate)//Only fired when the Chacontrol.NowCoordinate.loadfile is used
    {        
        var plugindata = GetCoordinateExtendedData(coordinate);
        if (plugindata == null) return;
        if (plugindata.data.TryGetValue("Example", out var ByteData))
            Now_Coord_Data = MessagePackSerializer.Deserialize<Example_Class>((byte[])ByteData);
    }
 }

[Serializable]
[MessagePackObject]
public class Example_Class
{
    [Key("_exampleint")]
    public int Exampleint;
    
    [Key("_examplestring")]
    public string Examplestring;
    
    [IgnoreMember]//not serialized
    public int ignoredint;
    
    public Example_Class(){}
    
    public Example_Class(int _exampleint, string _examplestring)//when deserializing will look for constructor with matching keys
    {
        Exampleint = _exampleint;
        Examplestring = _examplestring;
    }
}

ExtendedSave: Previous Generic Standard VS. New Standard

There is a new method for saving data to cards. For now lets call the previous standard the generic standard. The generic standard itself is simply tied to the card itself and not any specific part of it. It was a simple and straight forward method of saving card data. The downside is that the data is completely seperate from any part that it is associated with. For example data associated with coordinate 0 will not transfer automatically if it becomes coordinate 1. The new standard will attempt to address this.

Full New Standard Extentions

The goal of the new standard is remove some data handling from plugin specially when it concerns the movement of data and should remove some overhead memory cost as a result at the cost of deserializing more often when coordinate changes.

Loading and Saving for the new standard is managed by using new extentions provided by using static ExtensibleSaveFormat.Extensions. There are multiple individual components that can have data saved to such as the Body, Pupil, Clothes Set, individual accessories and more.

The only non-straight forward method required is for when dealing with individual accessories. The reasoning being that at the very least there are 20 to deal with and if MoreAccessories is included some coordinates might have hundreds to loop through. As such it would be best to have a list of integers representing slot numbers saved to the Clothes Set (ChaFileClothes) as that is effectively bound to the coordinate at most.
> not sure if this is actually good advice while it would indeed reduce the amount of deserializing, but adjusting this is still less work when moving stuff around(jalil)

Contributing to the API

Because an API's functions are publically available it becomes possible to eventually bloat the API with outdated methods and classes. Ideally when contributing to API it's best for contributed code to be widely applicable as possible for public facing code.

– common patterns and reasonings
class and file naming, plugin/controller/hooks
guid and version const
assembly attribs, file version and url
project naming for multiple games
nuget over direct dlls
publicized assemblies, avoid reflection where possible
harmony hooking, prefer patchall, sanity checking inside transpliers and codematcher
wrap hooks and event handlers in trycatch (exceptions - extsave, transplier)
sanity checks everywhere to make debugging easier, error reports
code style, resharper
use api plugins instead of copying code into your plugin, fixes happen and easier to support new games

Creating a Preloader patcher

Doorstop is the provided entrypoint for BepInEx initialization. It runs code before Unity is fully initialized. During BepInEx's startup routines, there is an opportunity to use Cecil modify assemblies as they are loaded.
This has the advantage of achieving what can otherwise not be done through Harmony patches.
For example:

  • Injecting new fields or attributes into a class
  • Replacing entire assemblies
  • Modifying assembly references

Cecil exists for the edge cases where Harmony and MonoMod can't quite reach. These two libraries are better suited to modifying existing behaviour and control flow whereas Cecil treats the assembly itself as a structure to be modified.

Preloader patchers target assemblies and these are passed to a patcher method which can modify them in-place. These modifications then persist for the duration of the application's execution.

using System.Collections.Generic;
using Mono.Cecil;

public static class Patcher
{
    // List of assemblies to patch
    public static IEnumerable<string> TargetDLLs { get; } = new[] {"Assembly-CSharp.dll"};

    // Patches the assemblies
    public static void Patch(AssemblyDefinition assembly)
    {
        // Patcher code here
        //Each targetted DLL is passed in as a parameter
    }
}

For more information about writing Preloader patchers

Cecil usage is fairly complex, it is recommended to read the docs

Prototyping plugins and patchers

  • dnspy hardmod
  • runtime editor repl
  • script engine
  • script loader

Supporting multiple games with one codebase

– why share code not separate plugins, keeping all games up to date
– when a single plugin can run on many games and when not
referencing only unity dlls
making the plugin run on as many versions of unity as possible
why it might still be a good idea to make per game version + universal version, having unique tweaks like resourceunloadopt plugin
– ways of sharing code between games
shared projects, inheritance, referencing individual files
use api plugins whenever possible to reduce code unique per game
– making a KK-only plugin work in KKS as well step by step
see how much of the plugin works in the new game
choose an approach, will use a shared project
head projects for each game
– kk compatibility analyzer, how to deal with differences in darkness and non-darkness
how to check and access members that only exist in some games. use reflection/wrap in methods or it will crash

Porting a plugin to a new game

It's best to share as much code as possible between versions for different games. Ideally almost all generic code should be put inside of a shared project, and only game-specific hooks and features should be in the specific game projects. This will ensure that all games are kept up to date and with even features.

If you no longer want to support one of the games, e.g. because it's too old and prevents you from using new features, make sure to conserve the code for this old game!

  • You can either create a "legacy" branch (You can then remove the game-specific code from your main branch),
  • or you can copy everything from the shared project into the game-specific project and remove the shared project reference (This way the old game project has a known good snapshot of the shared files. This lets you easily add fixes to the old game without worrying about breaking it).

Strategy for porting a single-game plugin to a new game

This is an example strategy that works well for me (Marco here, hello!).

  1. Create a new project for the new game. Add packages and/or assembly references needed by the game.
  2. Set up conditional compilation symbols in both projects (Project properties > Build). For example for Koikatsu add a KK symbol. Make sure to add it to all configurations.
  3. Copy contents of the old project to the new project, so you have everything duplicated.
  4. Modify code in the new project until it works in the new game.
  5. Make a commit with your changes so you have a safe checkpoint.
  6. Use a diff tool like WinMerge to compare old project with the new project. You should now see all of the changes you've made to port the plugin.
  7. Merge the changes so that at the end both new project and old project code is as identical as possible. Use conditional compilation symbols in places where old and new game code is different.
  8. Create a new shared project and reference it in your existing projects.
  9. Move all of the files that you merged in step 7 to the shared project (the ones that you managed to make identical). If any of the files have too many changes to merge them cleanly, leave these files in the separate game projects.
  10. Remember you can also move the AssemblyInfo file to the shared project. Make sure the version number is set inside the shared folder so all game versions get bumped at the same time.

Debugging plugins and game code

– what and why
–- saving time
–- making it easier by adding debug stuff to code
– approaches
–- logging, dnspy, unity/vs
dnspy can debug game startup/patchers, other have to connect midway
– step by step dnspy break inside your plugin and change a variable

Profiling

What is it and why do I want it

Profiling is the act of measuring code performance in order to find performance issues in it. This knowledge can then be used to optimize it in order to improve game FPS, loading speeds, reduce stutter, speed up networking, etc.

We are interested in badly optimized code that can cause one of the two: low FPS or stuttering. Both of these are obviously not desirable, and the more plugins are added to the game the worse the performance hit will get if not properly managed.

Illusion games tend to be CPU limited, so slow code will usually directly translate into lower FPS. That being said, if you have a fast CPU and a very slow GPU (for example integrated graphics) then your FPS might be limited by the GPU, which means that you might not see slow code have any effect on FPS because your CPU doesn't work that hard and has some headroom.

The two types of bad perfomance have very different causes and effects:

  • Low FPS is caused by running a lot of instructions every frame. It makes the game feel less smooth and is tolerable to some degree.
  • Stutter is caused by the code allocating and discarding a lot memory (garbage, stutter is caused by the garbage collector cleaning it up), or loading/unloading assets/files (resources, stutter is caused by I/O or searching the scene for unused assets). Even a little of stutter can make the game feel slow and laggy, even if the FPS is high.

Often, but not always, both of these are caused by the same pieces of code. Badly optimized code is usually slow because of creating a lot of new objects in a short amount of time - the solution is to eliminate or at least reduce creation of new objects. Another common cause of slow code are badly optimized linq queries and Contains searches in collections - common solutions are avoiding using complex queries, using lookup tables / hashsets / dictionaries instead of lists, changing foreach loops to for loops, etc.

Premature optimization

It's important to know when to optimize and when to do something else. Don't optimize your code unless you know for a fact that it will cause issues, either by experience or by using a profiler. Excessive optimization more often than not will have next to no effect on the overall performance and will be a waste of time.
Avoid microoptimizations (e.g. using byte instead of int, using for loops instead of foreach, looping manually instead of using linq) unless a profiler without a doubt tells you there's an issue.

Common performance issues

  • Running excessive amounts of code on every frame, especially expensive methods like Object.Find. Avoid using Update and LateUpdate if possible, and move as much of the code as possible into Start (i.e. find and cache objects for later use instead of searching for them every time).
  • Very inefficient algorithms. It's best to take at least some care to write efficient algorighms from the get-go (e.g. instead of searching a list for some value 100 times in a loop, make a lookup dictionary and look up by key).
  • Excessive logging - writing to log is surprisingly expensive, especially if you have the console open. The number of separate log calls is the issue, the actual number characters written doesn't matter much, so if you need to log a lot of information you should combine it into a single string and log it all at once.
  • Triggering expensive game methods, for example reloading the whole character when all you want to do is update the body texture. Look for a way to make the game update only the part you need.
  • Using a debug mono dll (disables incremenal GC and might be slightly slower) or any profiler (StartupProfiler only slightly slows down startup, FPSCounter adds overhead to all Update method calls, SimpleMonoProfiler adds overhead to all method calls).

Further reading

You can read more about allocating memory, garbage and garbage collection in C# and Unity here: https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/fundamentals
Note that Unity uses a custom mono build instead of CLR, so the details will be different. For example Unity does not have GC gens / compacting.

You can read more about profiling and optimization techniques here:

Incremental GC

New versions of Unity (2019.1 and above) support an incremental garbage collector that in large part alleviates stutter caused by garbage, but it comes at the price of lower FPS. Note that incremental GC is experimental (might have bugs) and is only available in KKS and newer games, and it might not be turned on in your game (KKS has it turned off). Long story short, don't rely on this to hide your badly optimized code, because it will still be slow, just less noticeably so.

To enable incremental GC add gc-max-time-slice=3 to boot.config. The game must support it for it to work. Incremental GC will not work if you have a debug mono dll for use with dnspy. To see if it works you can check GarbageCollector.isIncremental at runtime.

You might still get random stutter if there is too much garbage being generated to the point of overwheling the GC.

Read more: https://resources.unity.com/developer-tips/incremental-garbage-collector

Profiling tools

There are several different approaches we can take to debug Unity games. When modding a game we don't have access to the original project so our choices are somewhat limited - we can't run the game in editor and can't (easily) use the Unity's built-in profiler. There are some other ways to do it though:

  • FPSCounter - BepInEx plugin that tells you what parts of the game take the most time to render, and which plugins are being slow. It's a good way to get a rough idea of what is causing the most FPS drop.
  • BepInEx.Debug StartupProfiler - Used to measure how long each plugin takes to initialize, directly affecting boot time of the game. Results are written into the log.
  • BepInEx.Debug SimpleMonoProfiler - Used to measure time taken by each method to complete and garbage generated while it was running. Very powerful but more difficult to use effectively.
  • Unity Profiler - Unity's own built-in profiler. To use it you have to install Unity Editor of the same version as your game, and you have to turn your game into a debug build (varies how by Unity version, check this).
  • dnspy - While it's not an actual profiling tool, it will help you nail down performance issues inside the game code if the profiler points you at it. You can also use it to temporarily remove or change chunks of code to check if that improves performance without having to write a harmony hook.

What to look for

Run around with FPSCounter open and look for any FPS drops and excessive GC allocations. If most of the time is spent rendering the graphics you might need to turn down your resolution and picture quality. BepInEx console with All log levels turned on will be useful to determine causes of some stutter and lag, for example resource unloads or characters loading/reloading.

It's a good idea to test the game with no mods installed at all other than FPSCounter and maybe other profiling tools to get a feel for how things are supposed to run. This will give you a baseline to compare to and make spotting issues like long loading screens easier.

How to use SimpleMonoProfiler

  1. First, grab it from the debug tools repository and extract it into your game folder. It has two native dlls for 32 bit and 64 bit games, you can leave them both or remove the wrong one.
  2. The profiler is always active and will slow the game down quite considerably. To turn off the profiler you can rename the native dll.
  3. To write data captured so far to the drive and start a fresh capture, press the ` key (backquote, can be changed if needed). It will write a new .csv file in your game directory with the capture date in its filename.
  4. To open the captured .csv you can use LibreOffice Calc, Excel, or a similar spreadsheet app that can import csv files. Make sure that the columns are properly detected. I'll be using LibreOffice Calc for the instructions.
  5. Select the whole 1st row, then Data > Auto Filter. This will let you sort and filter by each of the columns.

Contents of the csv file

  • Thread refers to ID of the thread that the code runs on. Only code running on the main thread will slow the game FPS down (this might not work on some mono versions and always show 0).
  • Call count is how many times the method was called during the capture window. Higher count means more overhead, and most likely running every frame instead of once.
  • Method name is the best attempt at identifying the method. If you see Thread.Sleep then it's likely running on a background thread and can be ignored. Some methods might have weird names (e.g. DMD<UpdateDynamicBones>?-1420185344:_DynamicBone::UpdateDynamicBones (DynamicBone,single)) - those are patched by plugins. The entry will also count time spent in code added by these plugins. Some of the entries might not be a part of the source code, e.g. boxing.
  • Total runtime is how much time was spent inside the particular method across all calls. Generally you want to pick the highest numbers and look for a way to lower them down - this is the best way to improve FPS if the game is CPU-limited.
  • Total allocation is how much memory was allocated (not necessarily discarded), so look for high numbers if you want to reduce stuttering caused by garbage collections.

Warnings and pitfalls

  • Remember that runtime and allocation data is inclusive - it includes everything that happens inside that method, even if it's another methods. Those other methods will also appear on the list. You will want to find the method that is the origin (so it doesn't call any other methods that add up to its runtime or allocations).
  • Allocations are measured by comparing GC heap usage before and after the method finishes. This means it might include allocatons made in background threads. If possible, prevent background threads from running during the capture (coroutines are fine since they run on the main thread). If a garbage collection happens while inside the method, the capture will be discarded.
  • Results may differ if you use a debug mono dll. Use the original dll to get the best idea of end-user performance, and use the debug dll as more of a stress test.
  • The profiler will add some overhead to each method call, slowing the game down noticeably. Be careful with trivial methods that have very high call counts - they may be erroneously reported as taking massive amounts of time because of the profiler overhead being added to each call.

Profiling universal plugins inside Unity Editor

If your plugin doesn't rely on any game code, only basic UnityEngine functionality, you can make an empty test project in Unity Editor, build it as a debug project, and test your plugin(s) inside that. This will let you use the Unity Profiler in the "proper" way and reduce background noise in your measurements.

What if the issue is in someone else's code?

If they use github, you're in luck. If it's an easy fix, then do it and make a PR, I'm sure the maintainer will appreciate it. Check the git/github section for more info about how to do this.

If fixing it would be more involved then it's better to make an issue first to talk it over. Even better if you can talk about it on an instant messaging platform like Discord.

What if the issue is in game code?

Assuming you don't have access to the game code to fix and recompile, you will have to patch it at runtime with harmony or cecil. Generally it's best to do it with harmony. You can see an example plugin that optimizes the game by reducing unnecessary allocations here.

Git, GitHub and plugin hosting

What is what?

​​​​collab and more trustworthy
​​​​how to encourage others to contribute, keep code clean and add a readme, avoid reimplementing 
​​​​github vs other sites, effect on contributions
​​​​backups

making a new repo and pushing your plugin to it step by step

–- clone with different tools

contributing to an existing plugin step by step

–- simple gui edit
–- make a fork and push to it
–- PR and issues, why pr is better and how to make good issues

content/asset mods / zipmods/hardmods

  • introduction
    – what can they do
    – hard vs soft
    –- sideloader - what it really is and how does it work
    –- hardmods - what they are good for (dev), never share with people
    – skills required
    –- where else the skills are useful
    –- difference between normal game making in unity editor
    – what games it applies to
    – software required, unity engine correct version, sb3u
    –- getting and installing
    –- making mods in unity vs sb3u
    –- mod tools
    –- bepinex install and enabling console, why important
    – hosting zipmods, sideloader modpacks
    – example zipmods and stuff

  • list files
    – what are they
    – file structure
    – why most difficult part
    – how to learn how they work

  • zipmod format
    – how to load and use, sideloader basics
    – overall file structure
    –- why store compression, copress bundles instead
    – manifests
    –- custom manifest data
    –– uncensorselector
    –– clothingcolliders
    –– ME
    –– others

  • making a studio? item zipmod
    – whats needed
    –- unity vs sb3u
    –- mod tools
    – step by step in untiy editor
    – step by step in sb3u
    – manifest and packing a zipmod, store compression and why, compress bundle instead but not lzma

  • making a clothing item zipmod?

  • making an ME shader

  • making an US uncensor
    – how to make it BP compatible

Contributing

  • who made this, how to contact
  • how to edit this guide
Last changed by 
 
ManlyMarco
https://github.com/ManlyMarco https://www.patreon.com/ManlyMarco
3
4189

Read more

Basics of using StableDiffusion, or "How to make your Koikatsu pictures look arguably better"

This guide is meant for beginners that want to try generating your own anime pictures or modifying existing pictures, e.g. KK screenshots. It will guide you through installing a local copy of WebUI and some SD (stable diffusion) models. Word of caution: you will need* a beefy NVIDIA GPU, details below. Preview of a KK screenshot being redrawn in anime style Example of what can be accomplished with this guide (% refers to how much the AI was allowed to redraw, roughly speaking). First, some basics: SD / Stable Diffusion is a diffusion model. Diffusion Models are generative models, meaning that they are used to generate data similar to the data on which they are trained.

Mar 26, 2023
Read more from ManlyMarco
Published on HackMD