Performance vs flexibility in a Unity Node Graph Editor Tool

Question

I'm developing a node-based editor tool in Unity with pure UIToolkit (no existing graph frameworks). The premise is that the graph needs to be scene-based, so it's a Monobehaviour, rather than a ScriptableObject, as most other graph systems do. Inside each node, there can be PropertyFields that potentially draw a custom property UI.

I've implemented two approaches, both with their drawbacks and benefits.

1. In memory ScriptableObjects for the Graph and the Nodes, like so:

    public class GraphComponent : MonoBehaviour
    {
        [SerializeReference]
        [HideInInspector]
        public GraphData graphData;

        ...

    public class GraphData : ScriptableObject
    {
        [HideInInspector] public string GUID = Guid.NewGuid().ToString();

        [HideInInspector]
        [SerializeReference]
        public List<NodeData> nodes = new();

        ...

    public abstract class NodeData : ScriptableObject
    {
        [NodeHideVariable]
        public string guid = "";

        [SerializeReference]
        public GraphData graphData;

        ...

Then of course there are lots of ViualElement derived classed that define how nodes, edges etc. look.

In the NodeElement class there are methods to build a nodes VisualElement hierarchy, the most important one being BuildValues() iterating over the fields and creating PropertyFields, roughly like so:

        public void BuildValues()
        {
            var _editor = Editor.CreateEditor(nodeData);
            var sp = _editor.serializedObject.GetIterator();

            sp.NextVisible(true);
            while (sp.NextVisible(false))
            {
                System.Type t = sp.serializedObject.targetObject.GetType();
                
                FieldInfo f = null;
                f = t.GetField(sp.propertyPath);
                if (f != null)
                {
                    var _hideAttribute = f.GetCustomAttribute(typeof(NodeHideVariable), true);
                    if (_hideAttribute == null)
                    {
                        var _property = new PropertyField();
                        _property.BindProperty(sp);

                        valuesContent.Add(_property);
                    }
                }
            }
            _fromNodeInspector = false;
            
            _editor.serializedObject.ApplyModifiedProperties();
        }

This approach runs extremely efficient and easily handles graphs with 200 nodes and more, with a butter smooth framerate in the editor. Absolute joy to work with. It serialized the in-memory ScriptableObjects to the Unity scene as expected. The catch: It obviously doesn't work if you want to make a GraphComponent a prefab, by dragging it into the Project panel. The graphData will contain no nodes whatsoever, as references to ScriptableObjects are not serialized to prefabs in Unity. So I'd have to dive deeply into a rabbithole of generating ScriptableObject subassets for each node and somehow manually resolving the internal references to scene objects, that are within the Prefab hierarchy (if that's even possible).

So I tried a second approach:

2. Using System.Serializable classes for GraphData and NodeData, like so:

GraphComponent stays unchanged.

    [System.Serializable]
    public class GraphData
    {
        [HideInInspector]
        [SerializeReference]
        public MonoBehaviour monoBehaviour; // store to which GraphComponent we are attached
        
        [HideInInspector] public string GUID = Guid.NewGuid().ToString();
        
        [HideInInspector]
        [SerializeReference]
        public List<NodeData> nodes = new();
        
        ...

    [System.Serializable]
    public abstract class NodeData
    {
        [NodeHideVariable]
        public string guid = "";
        
        [SerializeReference]
        public GraphData graphData;
        
        ...

The new BuildValues() also iterates over the GraphComponent as a SerializedObject and binds PorpertyFields to its SerializedProperties down the serialization chain:

        public void BuildValues() {
            SerializedObject serializedObject = new SerializedObject(graphData.monoBehaviour);
            serializedObject.Update();

            // Find GraphData property
            SerializedProperty graphDataProperty = serializedObject.FindProperty("graphData");

            // Find nodes list
            SerializedProperty nodesProperty = graphDataProperty.FindPropertyRelative("nodes");

            // Find index of nodeData in nodes list by comparing GUIDs
            int nodeIndex = -1;
            for (int i = 0; i < nodesProperty.arraySize; i++)
            {
                SerializedProperty nodeProperty = nodesProperty.GetArrayElementAtIndex(i);
                SerializedProperty guidProperty = nodeProperty.FindPropertyRelative("guid");
                if (guidProperty != null && guidProperty.stringValue == nodeData.guid)
                {
                    nodeIndex = i;
                    break;
                }
            }

            if (nodeIndex == -1)
            {
                Debug.LogError("NodeData not found in GraphData nodes list.");
                return;
            }
            // Get SerializedProperty for nodeData
            SerializedProperty nodeDataProperty = nodesProperty.GetArrayElementAtIndex(nodeIndex);

            // Clear existing content
            valuesContent.Clear();

            // Iterate over nodeData fields
            SerializedProperty iterator = nodeDataProperty.Copy();
            SerializedProperty endProperty = nodeDataProperty.GetEndProperty();

            // Move into the first child of nodeDataProperty
            if (iterator.Next(true))
            {
                int startingDepth = iterator.depth; // Record the starting depth

                do
                {
                    // Stop if we've reached the end property or exited the current depth
                    if (SerializedProperty.EqualContents(iterator, endProperty) || iterator.depth != startingDepth)
                        break;
                    
                    // Get the corresponding FieldInfo
                    FieldInfo fieldInfo = nodeData.GetType().GetField(iterator.name,
                        BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.FlattenHierarchy);

                    if (fieldInfo == null) continue;

                    // Create PropertyField
                    PropertyField propertyField = new PropertyField(iterator.Copy());
                    propertyField.Bind(serializedObject);
                    valuesContent.Add(propertyField);
                } while (iterator.NextVisible(false));
            }

            // Apply changes when needed
            serializedObject.ApplyModifiedProperties();
        }

This approach works perfectly and naturally with Prefabs, but as soon as you have more than 30-35 nodes it lags like crazy and makes the graph tool unusable. From Profiler runs I found that because the PropertyFields are bound to the higher level Monobehaviour, Unity will serialize the whole structure with every little change. So, e.g., when ou move a single node, all other nodes are updated and rendered. I tried to be smart and wrap the NodeData Objects into a NodeDataWrapper class (derived from ScriptableObject) that only holds a reference to the same NodeData, and bind the PropertyFields to those ScriptableObject wrappers instead, but no difference in performance.

So the question is: can this latter approach be made efficient in rendering, while using System.Serializable classes to maintain Prefab workflow compatibility?

Update: I also tried a custom property drawer for my NodeData objects so Unity would auto-bind to the properties but the performance is still laggy and no different than before.