本课程深入讲解策略梯度与PPO，专攻四足机器人、无人机的连续控制，涵盖奖励设计、奖励作弊及Sim-to-Real转换，通过领域随机化解决仿真现实差距，适合机器人及ML工程师提升实战能力。
原始标题：Deep RL for Robotics: Continuous Control, Locomotion & Sim
本课程专注于四足机器人和无人机等连续控制任务，从零推导策略梯度与PPO公式，并通过重现“奖励作弊”现象深入讲解奖励设计与Sim-to-Real控制，致力于缩小理论与实际硬件的差距。课程强调工程实践，指导如何利用领域随机化（Domain Randomization）在CPU环境下诊断与解决仿真到现实的转换问题，适合机器人及ML工程师攻克复杂环境下的机器人RL部署。
Published 8/2026
Created by Ferbin Richard
MP4 | Video: h264, 1920×1080 | Audio: AAC, 44.1 KHz, 2 Ch
Level: All Levels | Genre: eLearning | Language: English + subtitle | Duration: 106 Lectures ( 11h 41m ) | Size: 3.2 GB
Master robot RL: policy gradients, PPO, reward design and sim-to-real transfer for quadrupeds and drones
What you’ll learn
⚡ Derive the policy gradient theorem and PPO’s clipped objective from scratch, with every step worked on screen rather than quoted as a result.
⚡ Design reward functions for continuous control, and recognise reward hacking by reproducing it on a real simulated quadruped, quadrotor and tracked robot.
⚡ Diagnose why a policy that works in simulation fails on hardware, and apply domain randomisation to close the reality gap.
⚡ Read and debug a real PPO implementation: advantage estimation, parallel environments, and the implementation details that decide whether it trains at all.
Requirements
❗ Comfortable reading Python.
❗ Basic calculus and linear algebra
❗ No reinforcement learning background needed.
❗ Helpful but not required: exposure to classical control (PID) or robotics. If you have used CartPole or Atari RL before, expect those intuitions to be challenged.
Description
This course contains the use of artificial intelligence.
Most reinforcement learning courses teach you CartPole. Then you try to apply it to a robot and discover that almost nothing transfers.
This course starts where those stop. Every example is a quadruped, a quadrotor or a tracked ground robot. Every action is a continuous torque or thrust carrying real units and real limits — never a
discrete button press. If you have trained an agent to balance a pole and felt no closer to controlling a machine, this is the course that closes that gap.
Every derivation is worked on screen. The policy gradient theorem is built term by term, not quoted. Generalised advantage estimation is derived, not cited. PPO’s clipped objective is assembled
piece by piece, and then we turn the clipping off to show exactly what it was preventing. You will finish able to read a real PPO implementation and know what each line is defending against.
Reward design gets a full section, because it is where robot RL actually fails. You will watch reward hacking reproduced rather than described: a velocity reward satisfied by vibrating in place, a
height reward satisfied by flipping onto its back with feet in the air, a goal reward satisfied by knocking the goal over. None of these are bugs in the algorithm. Each is the algorithm doing
precisely what the number told it to.
Two full sections anatomise the reality gap — why a policy that walks perfectly in simulation falls over on real hardware, and what domain randomisation can and cannot do about it.
One honest limit, stated up front: this course teaches the reality gap without crossing it. There is no physical robot. Every “real robot” on screen is a deliberately mismatched simulation, and it
is labelled as such every time.
Nothing is installed. No GPU. No hardware. Every notebook is CPU-only and runs top to bottom on a consumer laptop.
You will learn to: derive policy gradients and PPO from scratch · design rewards for continuous control and recognise when they are being gamed · diagnose sim-to-real failure · apply domain
randomisation · read and reason about a real PPO implementation.
Who this course is for
⭐ Robotics engineers who know classical control and want to understand what reinforcement learning adds, and where it genuinely does not belong.
⭐ ML engineers and researchers who have done RL on CartPole, Atari or gridworlds and now need continuous torque control on real robot morphologies.
⭐ Graduate students and self-taught learners preparing for locomotion, sim-to-real or Isaac Lab work, who want the derivations rather than a library tutorial.
此处内容需要权限查看
会员免费查看
