Satellite Attitude Control using Reinforcement Learning and State Space Model

Abstract

This paper presents a reinforcement learning (RL) based framework for satellite attitude control using a state-space model. The Proximal Policy Optimization (PPO) algorithm is used to train an agent for three-axis satellite reorientation in a simulation environment governed by Euler's rotational equations and quaternion kinematics. Real-world Inertial Measurement Unit (IMU) data collected from Micro-Electro-Mechanical Systems (MEMS) Accelerometer and Gyro sensors was used to characterize noise parameters and validate simulation fidelity. The trained PPO agent was evaluated against an untrained baseline and a cascade Proportional-Integral-Derivative (PID) controller over 500 randomized episodes. The trained RL agent achieved a 96% success rate with 18.3° mean pointing error and 98.8% alignment score, closely competitive with PID which achieved 100% success rate, 26.0° mean pointing error, and 99.8% alignment score, while using comparable control effort 276 Rate Per Minute (RPM) for RL agent vs 287 RPM for PID. A 3D interactive visualization system was developed for real-time trajectory inspection. Results confirm the feasibility of RL-based attitude control and identify clear directions for improvement.