Simple Video Codec for the Nintendo DS11/05/2022
I had an idea today to try to create a simple video codec specific to the Nintendo DS that used its native hardware features to try to achieve a good compression ratio. The idea is to use 4x4 compression as a basis, since it can effectively store pixels at a rate of 3 bits per pixel. Not only this, but the picture quality is still quite good if compressed well. On top of this, I LZ compress the texel and index data of each frame, which reduces their size by a noticeable amount.
The resulting picture quality is okay, but the file sizes are abysmal. A 90-frame video takes 837KB to store, which is a bit much for my liking. What makes me happy though is that it can still play back the video at a smooth 60 frames per second.
Different measures will need to be taken to try to improve the compession ratio while not taking a hit to performance. My main fear with this is that with the LZ compression, there may not be very much wiggle room left for operations that can be done on the frame data to get it sent to VRAM in time. Either the other code will have to be really fast, or maybe LZ compression will have to be replaced with something a little less intense at runtime, perhaps something more geared towards 4x4 texture data specifically.
I talked to Gericom a bit about it, and he suggested that I try to use the ARM7 to offload some of the processing, which I really should have thought of sooner, being one of the DS's more distinctive features. Since the ARM7's bus timings for non-sequential memory access aren't broken like the ARM9's, it could improve the time taken to perform the decompressions. Another idea tossed around was to use layers, which could be used to approximate the concept of I-frames and P-frames, which currently do not exist in my implementation. These have the potential to reduce the file size a fair bit combined with the lossless compression stage on top.
Additionally, some parts of the frame could be reused for other frames to limit the amount of potentially redundant data being decompressed and uploaded per frame. One scheme I thought of was sharing palettes and index data between frames, since most of the time objects are pretty similar from one frame to the next, it may not be worthwhile to have two entirely different sets of palette and index data. In theory this will reduce the file size as well as the decoding time. I'm not sure where I'll go with this yet, we'll see.