I’m thinking about adding support for streaming soon to GODZ. Being a multi-core engine, I can already stream content in another thread of course. For instance when the engine starts the render thread loops while the game thread handles loading the resources it needs (well it requests the Render thread to instantiate the object once the geometry data is loaded). The problem here- I want my Gameplay thread to focus on GAMEPLAY. It is okay the render pipeline handles creating my resources on the video card that cant be helped (directX is picky about requests coming to it from different threads). But Gameplay thread handling loading resources is Bad I have concluded. But at least I have some of this working.
Right now, Gameplay runs concurrently with Render thread. Game thread submits data for next thread.
So, if I want to stream content into the game I will have to—>
- Let Gameplay thread run concurrently with this ResourceStream thread. Perhaps the Game thread will request all the content it needs, while the resourceStream thread runs concurrently in the background, grabbing required resources (load them from disk in the background). Like most game engines, GODZ has a Serialize() function that can be invoked on objects. The ResourceStream will handle all serialization in the background.
- The Render thread will check for new resources being loaded up and attach those to the necessary gameplay objects as necessary (Ideally gameplay thread should only have an ID that references a render object but right now its a pointer, bad). This is where we might get a slight performance pause- because the renderer will need to process these requests right after it renders the frame. Recall, the DirectX9 API runs at its best when we are in single core mode on its end. So all calls to the API must be from the Render thread which owns the device. So this can’t be helped. Note- I have already done this entire step- the renderer already knows how to do its job actually. If I wanted to be smart about it, could split up the mesh creation steps across several frames (perhaps this is what most titles do).
- Wonder if I will need to utilize a “Future” to store a reference to a resource (note)
Content Side Production:
- Mapper can import a Level as usual but this level package will be serialized as a “Chunk”.
- During runtime, ResourceStream saves entry for every object loaded. When we receive a request to load a new Chunk we do not just simply dump everything but rather check to see what entries already exist and only load missing resources.
Looking at the source code in my engine it appears I made some architectural errors (poor judgement). I have a global array that stores a pointer to all of my objects for garbage collection. This global array is accessed by any thread whenever a new game object is created. Luckily, the render pipeline never does this so I never had a data race occur. But once I split off streaming code into a seperate thread I’ll run into some issues. sigh….. Global data is such a no-no in multi-core programs. I never did get around to looking Thread local storage to ensure game data cannot be accessed by Render pipeline. Unfortunately, I also used the Singleton programming pattern- which is no good for a multi-core architecture. Need to move these singletons to the Thread local storage level if I want a clean architecture…