AI Plays RE4 via BC + LSTM

🔑 Enhanced Key Takeaways

• •The project utilizes a custom-built frame-skipping mechanism to reduce the input dimensionality, allowing the LSTM to process temporal dependencies at 10 FPS rather than the native 60 FPS of the game.
• •The model architecture incorporates a pre-trained ResNet-18 backbone for feature extraction from raw pixel input before passing latent representations into the LSTM cell.
• •The training dataset consists of approximately 40 hours of human gameplay, specifically curated to include 'no-damage' runs to bias the model toward optimal pathing and resource management.

🛠️ Technical Deep Dive

• •Architecture: Hybrid CNN-LSTM model.
• •Input: 224x224 RGB frames captured via OBS virtual camera stream.
• •Output: Discrete action space mapping to controller inputs (Move, Aim, Shoot, Knife, Reload).
• •Loss Function: Cross-entropy loss for classification of discrete actions, combined with a small auxiliary loss for predicting the next frame (self-supervised learning).
• •Memory: Hidden state size of 512 units in the LSTM layer to maintain context of enemy positions off-screen.

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline