{"id":19314,"date":"2020-04-05T01:15:23","date_gmt":"2020-04-04T17:15:23","guid":{"rendered":"http:\/\/www.zhushiyao.com\/?p=19314"},"modified":"2020-04-05T01:18:18","modified_gmt":"2020-04-04T17:18:18","slug":"ce849df780","status":"publish","type":"post","link":"http:\/\/www.zhushiyao.com\/?p=19314","title":{"rendered":"\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0"},"content":{"rendered":"
\n

\u6765\u6e90\uff1a\u673a\u5668\u4e4b\u5fc3<\/span><\/p><\/blockquote>\n

\n
\n
\u8bad\u7ec3\u6a21\u578b\u7684\u6781\u901f\uff0c\u4e0e 1500 \u884c\u6e90\u4ee3\u7801\u7684\u7cbe\u7b80\uff0c\u6e05\u534e\u5927\u5b66\u65b0\u5f00\u6e90\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\u300c\u5929\u6388\u300d\u3002\u503c\u5f97\u6ce8\u610f\u7684\u662f\uff0c\u8be5\u9879\u76ee\u7684\u4e24\u4f4d\u4e3b\u8981\u4f5c\u8005\u76ee\u524d\u90fd\u662f\u6e05\u534e\u5927\u5b66\u7684\u672c\u79d1\u751f\u3002<\/span><\/section>\n<\/section>\n<\/blockquote>\n
\u662f\u5426\u4f60\u4e5f\u6709\u8fd9\u6837\u7684\u611f\u89c9\uff0c\u6210\u719f ML \u5de5\u5177\u7684\u6e90\u7801\u5f88\u96be\u61c2\uff0c\u5404\u79cd\u7ee7\u627f\u4e0e\u5904\u7406\u5173\u7cfb\u9700\u8981\u82b1\u5f88\u591a\u65f6\u95f4\u4e00\u70b9\u70b9\u7406\u6e05\u3002<\/span>\u5728\u6e05\u534e\u5927\u5b66\u5f00\u6e90\u7684\u300c\u5929\u6388\u300d\u9879\u76ee\u4e2d\uff0c\u5b83\u4ee5\u6781\u7b80\u7684\u4ee3\u7801\u5b9e\u73b0\u4e86\u5f88\u591a\u6781\u901f\u7684\u5f3a\u5316\u5b66\u4e60\u7b97\u6cd5\u3002\u91cd\u70b9\u662f\uff0c\u5929\u6388\u6846\u67b6\u7684\u6e90\u7801\u5f88\u5bb9\u6613\u61c2\uff0c\u4e0d\u4f1a\u6709\u592a\u590d\u6742\u7684\u903b\u8f91\u5173\u7cfb<\/strong><\/span>\u3002<\/span><\/section>\n
<\/section>\n
\u9879\u76ee\u5730\u5740\uff1a<\/span>https:\/\/github.com\/thu-ml\/tianshou<\/span><\/span><\/section>\n
<\/section>\n
\u5929\u6388\uff08Tianshou\uff09\u662f\u7eaf \u57fa\u4e8e PyTorch \u4ee3\u7801\u7684\u5f3a\u5316\u5b66\u4e60\u6846\u67b6\uff0c\u4e0e\u76ee\u524d\u73b0\u6709\u57fa\u4e8e TensorFlow \u7684\u5f3a\u5316\u5b66\u4e60\u5e93\u4e0d\u540c\uff0c\u5929\u6388\u7684\u7c7b\u7ee7\u627f\u5e76\u4e0d\u590d\u6742\uff0cAPI \u4e5f\u4e0d\u662f\u5f88\u7e41\u7410\u3002\u6700\u91cd\u8981\u7684\u662f\uff0c\u5929\u6388\u7684\u8bad\u7ec3\u901f\u5ea6\u975e\u5e38\u5feb\uff0c\u6211\u4eec\u8bd5\u7528 Pythonic \u7684 API \u5c31\u80fd\u5feb\u901f\u6784\u5efa\u4e0e\u8bad\u7ec3 RL \u667a\u80fd\u4f53\u3002<\/span><\/section>\n
<\/section>\n
\u76ee\u524d\u5929\u6388\u652f\u6301\u7684 RL \u7b97\u6cd5\u6709\u5982\u4e0b\u51e0\u79cd\uff1a<\/span><\/section>\n
<\/section>\n
    \n
  • \n

    Policy Gradient (PG)<\/span><\/p>\n<\/li>\n

  • \n

    Deep Q-Network (DQN)<\/span><\/p>\n<\/li>\n

  • \n

    Double DQN (DDQN) with n-step returns<\/span><\/p>\n<\/li>\n

  • \n

    Advantage Actor-Critic (A2C)<\/span><\/p>\n<\/li>\n

  • \n

    Deep Deterministic Policy Gradient (DDPG)<\/span><\/p>\n<\/li>\n

  • \n

    Proximal Policy Optimization (PPO)<\/span><\/p>\n<\/li>\n

  • \n

    Twin Delayed DDPG (TD3)<\/span><\/p>\n<\/li>\n

  • \n

    Soft Actor-Critic (SAC)<\/span><\/p>\n<\/li>\n<\/ul>\n

    <\/section>\n
    \u53e6\u5916\uff0c\u5bf9\u4e8e\u4ee5\u4e0a\u4ee3\u7801\u5929\u6388\u8fd8\u652f\u6301\u5e76\u884c\u6536\u96c6\u6837\u672c\uff0c\u5e76\u4e14\u6240\u6709\u7b97\u6cd5\u5747\u7edf\u4e00\u6539\u5199\u4e3a\u57fa\u4e8e replay-buffer \u7684\u5f62\u5f0f\u3002<\/span><\/section>\n
    <\/section>\n
    \u901f\u5ea6\u4e0e\u8f7b\u91cf\uff1a\u300c\u5929\u6388\u300d\u7684\u7075\u9b42<\/span><\/strong><\/span><\/section>\n
    <\/section>\n
    \u5929\u6388\u65e8\u5728\u63d0\u4f9b\u4e00\u4e2a\u9ad8\u901f\u3001\u8f7b\u91cf\u5316\u7684 RL \u5f00\u6e90\u5e73\u53f0\u3002\u4e0b\u56fe\u4e3a\u5929\u6388\u4e0e\u5404\u5927\u77e5\u540d RL \u5f00\u6e90\u5e73\u53f0\u5728 CartPole \u4e0e Pendulum \u73af\u5883\u4e0b\u7684\u901f\u5ea6\u5bf9\u6bd4\u3002\u6240\u6709\u4ee3\u7801\u5747\u5728\u914d\u7f6e\u4e3a i7-8750H + GTX1060 \u7684\u540c\u4e00\u53f0\u7b14\u8bb0\u672c\u7535\u8111\u4e0a\u8fdb\u884c\u6d4b\u8bd5\u3002\u503c\u5f97\u6ce8\u610f\u7684\u662f\uff0c\u5929\u6388\u5b9e\u73b0\u7684 VPG\uff08vanilla policy gradient\uff09\u7b97\u6cd5\u5728 CartPole-v0 \u4efb\u52a1\u4e2d\uff0c\u8bad\u7ec3\u7528\u65f6\u4ec5\u4e3a 3 \u79d2\u3002<\/span><\/section>\n
    <\/section>\n

    \"\u6765\u81ea\u672c\u79d1\u751f\u7684\u66b4\u51fb\uff1a\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0\"<\/p>\n

    <\/section>\n
    \u4ee5\u4e0a\u6d4b\u8bd5\u4f7f\u7528\u4e86 10 \u4e2a\u4e0d\u540c\u7684 seed\u3002CartPole \u548c Pendulum \u4efb\u52a1\u4e2d\u7684\u7d2f\u79ef\u5956\u8d4f\u9608\u503c\u5206\u522b\u8bbe\u7f6e\u4e3a 195.0 \u4e0e-250.0\u3002\u53ef\u80fd\u4f1a\u6709\u8bfb\u8005\u611f\u89c9\u8fd9\u4e24\u4e2a\u4efb\u52a1\u6bd4\u8f83\u7b80\u5355\uff0c\u4e0d\u592a\u80fd\u7a81\u51fa\u6846\u67b6\u7684\u4f18\u52bf\u3002\u8be5\u9879\u76ee\u4e5f\u8868\u793a\uff0c\u5728\u8fd9\u51e0\u5929\u5185\uff0c\u4ed6\u4eec\u4f1a\u66f4\u65b0\u5929\u6388\u5728 Atari Pong \/ Mujoco \u4efb\u52a1\u4e0a\u7684\u6027\u80fd\u3002<\/span><\/section>\n
    <\/section>\n
    \u5929\u6388\uff0c\u53ea\u9700 1500 \u884c\u4ee3\u7801<\/span><\/strong><\/span><\/section>\n
    <\/section>\n
    \u975e\u5e38\u4ee4\u4eba\u60ca\u8bb6\u7684\u662f\uff0c\u5929\u6388\u5e73\u53f0\u6574\u4f53\u4ee3\u7801\u91cf\u4e0d\u5230 1500 \u884c\uff0c\u5176\u5b9e\u73b0\u7684 RL \u7b97\u6cd5\u5927\u591a\u6570\u90fd\u5c11\u4e8e\u767e\u884c\u4ee3\u7801\u3002\u5355\u4ece\u6570\u91cf\u4e0a\u6765\u8bf4\uff0c\u8fd9\u6837\u7684\u4ee3\u7801\u91cf\u5df2\u7ecf\u975e\u5e38\u7cbe\u7b80\u4e86\uff0c\u5404\u79cd\u7c7b\u4e0e\u51fd\u6570\u4e4b\u95f4\u7684\u5173\u7cfb\u5e94\u8be5\u4e5f\u5bb9\u6613\u628a\u63e1\u4f4f\u3002<\/span><\/section>\n
    <\/section>\n
    \u9879\u76ee\u8868\u793a\uff0c\u5929\u6388\u867d\u7136\u4ee3\u7801\u91cf\u5c11\uff0c\u4f46\u53ef\u8bfb\u6027\u5e76\u4e0d\u4f1a\u6709\u635f\u5931\u3002\u6211\u4eec\u53ef\u4ee5\u5feb\u901f\u6d4f\u89c8\u6574\u4e2a\u6846\u67b6\uff0c\u5e76\u7406\u89e3\u8fd0\u884c\u7684\u6d41\u7a0b\u4e0e\u7b56\u7565\u5230\u5e95\u662f\u4ec0\u4e48\u6837\u7684\u3002\u8be5\u9879\u76ee\u63d0\u4f9b\u4e86\u5f88\u591a\u7075\u6d3b\u7684 API\uff0c\u4f8b\u5982\u53ef\u4ee5\u4fbf\u6377\u5730\u4f7f\u7528\u5982\u4e0b\u4ee3\u7801\u4ee4\u7b56\u7565\u4e0e\u73af\u5883\u4ea4\u4e92 n \u6b65\uff1a<\/span><\/section>\n
    <\/section>\n
    result<\/span>\u00a0=\u00a0collector.collect(n_step=n)<\/code><\/pre>\n
    <\/section>\n
    \u6216\u8005\uff0c\u5982\u679c\u4f60\u60f3\u901a\u8fc7\u91c7\u6837\u7684\u6279\u91cf\u6570\u636e\u8bad\u7ec3\u7ed9\u5b9a\u7684\u7b56\u7565\uff0c\u53ef\u4ee5\u8fd9\u6837\u5199\uff1a<\/span><\/section>\n
    <\/section>\n
    result<\/span>\u00a0=\u00a0policy.learn(collector.sample(batch_size))<\/code><\/pre>\n
    <\/section>\n
    \u6b63\u662f\u901a\u8fc7\u5927\u91cf\u7cbe\u7b80\u7684 API \u6784\u9020 RL \u6a21\u578b\uff0c\u5929\u6388\u624d\u80fd\u4fdd\u6301\u5728 1500 \u884c\u4ee3\u7801\u5185\u3002\u4f8b\u5982\u6211\u4eec\u53ef\u4ee5\u770b\u770b DQN \u7684\u6a21\u578b\u4ee3\u7801\uff0c\u5b83\u662f\u975e\u5e38\u6d41\u884c\u7684\u4e00\u79cd\u5f3a\u5316\u5b66\u4e60\u6a21\u578b\uff0c\u5728\u5929\u6388\u5185\u90e8\uff0cDQN \u6a21\u578b\u771f\u7684\u53ea\u7528\u4e86 99 \u884c\u4ee3\u7801\u5c31\u5b8c\u6210\u4e86\u3002\u5f53\u7136\uff0c\u8fd9 99 \u884c\u4ee3\u7801\u662f\u4e0d\u5305\u542b\u5176\u5b83\u516c\u7528\u4ee3\u7801\u5757\u7684\u3002<\/span><\/section>\n
    <\/section>\n
    \u5982\u4e0b\u4e3a DQN \u7684\u4e3b\u8981\u4ee3\u7801\u7ed3\u6784\uff0c\u6211\u4eec\u7701\u7565\u4e86\u90e8\u5206\u5177\u4f53\u4ee3\u7801\uff0c\u5404\u4e2a RL \u7b56\u7565\u90fd\u4f1a\u7ee7\u627f\u57fa\u672c\u7c7b\u7684\u7ed3\u6784\uff0c\u7136\u540e\u91cd\u5199\u5c31\u591f\u4e86\u3002\u53ef\u4ee5\u53d1\u73b0\uff0c\u5728\u5e38\u89c4\u5730\u5b9a\u4e49\u597d\u6a21\u578b\u540e\uff0c\u4f20\u5165\u8fd9\u4e2a\u7c7b\u5c31\u80fd\u521b\u5efa\u7b56\u7565\u3002DQN \u7b56\u7565\u7684\u5404\u79cd\u64cd\u4f5c\u90fd\u4f1a\u5199\u5728\u4e00\u8d77\uff0c\u540e\u7eed\u914d\u7f6e Collector \u540e\u5c31\u80fd\u76f4\u63a5\u8bad\u7ec3\u3002<\/span><\/section>\n
    <\/section>\n
    \"\u6765\u81ea\u672c\u79d1\u751f\u7684\u66b4\u51fb\uff1a\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0\"<\/p>\n
    <\/section>\n
    \u9879\u76ee\u4f5c\u8005\u628a\u6240\u6709\u7b56\u7565\u7b97\u6cd5\u90fd\u6a21\u5757\u5316\u4e3a 4 \u90e8\u5206\uff1a<\/span><\/section>\n
    <\/section>\n
      \n
    • \n
      __init__\uff1a\u521d\u59cb\u5316\u7b56\u7565<\/span><\/p>\n<\/li>\n
    • \n
      process_fn\uff1a\u4ece replay buffer \u4e2d\u5904\u7406\u6570\u636e<\/span><\/p>\n<\/li>\n
    • \n
      __call__\uff1a\u7ed9\u5b9a\u73af\u5883\u89c2\u5bdf\u7ed3\u679c\u8ba1\u7b97\u5bf9\u5e94\u884c\u52a8<\/span><\/p>\n<\/li>\n
    • \n
      learn\uff1a\u7ed9\u5b9a\u6279\u91cf\u6570\u636e\u5b66\u4e60\u7b56\u7565<\/span><\/p>\n<\/li>\n<\/ul>\n
      <\/section>\n
      \u5b9e\u9645\u4f53\u9a8c<\/span><\/strong><\/span><\/section>\n
      <\/section>\n
      \u5929\u6388\u5f88\u5bb9\u6613\u5b89\u88c5\uff0c\u76f4\u63a5\u8fd0\u884c\u300cpip install tianshou\u300d\u5c31\u53ef\u4ee5\u3002\u4e0b\u9762\u6211\u4eec\u5c06\u8be5\u9879\u76ee\u514b\u9686\u5230\u672c\u5730\uff0c\u5b9e\u9645\u6d4b\u8bd5\u4e00\u4e0b\u3002<\/span><\/section>\n
      <\/section>\n
      !git\u00a0clone\u00a0https:\/\/github.com\/thu-ml\/tianshou<\/span>
      !pip3\u00a0install\u00a0tianshou
      import<\/span>\u00a0os
      os.chdir('tianshou'<\/span>)<\/code><\/pre>\n
      <\/section>\n
      \u8be5\u9879\u76ee\u5728 test \u6587\u4ef6\u5939\u4e0b\u63d0\u4f9b\u4e86\u8bf8\u591a\u7b97\u6cd5\u7684\u6d4b\u8bd5\u793a\u4f8b\uff0c\u4e0b\u9762\u6211\u4eec\u5728 CartPole \u4efb\u52a1\u4e0b\u9010\u4e2a\u6d4b\u8bd5\u4e00\u756a\u3002<\/span><\/section>\n
      <\/section>\n
      !python\u00a0test<\/span>\/discrete\/test_pg.py
      <\/code><\/pre>\n
      \n
      \"\u6765\u81ea\u672c\u79d1\u751f\u7684\u66b4\u51fb\uff1a\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0\"<\/p>\n
      <\/section>\n
      !python\u00a0test<\/span>\/discrete\/test_ppo.py
      <\/code><\/pre>\n
      \n
      \"\u6765\u81ea\u672c\u79d1\u751f\u7684\u66b4\u51fb\uff1a\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0\"<\/p>\n
      <\/section>\n
      !python\u00a0test<\/span>\/discrete\/test_a2c.py<\/code><\/pre>\n
      <\/section>\n
      \"\u6765\u81ea\u672c\u79d1\u751f\u7684\u66b4\u51fb\uff1a\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0\"<\/p>\n
      <\/section>\n
      !python\u00a0test<\/span>\/discrete\/test_dqn.py<\/code><\/pre>\n
      <\/section>\n
      \"\u6765\u81ea\u672c\u79d1\u751f\u7684\u66b4\u51fb\uff1a\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0\"<\/p>\n
      <\/section>\n
      \u4ee5\u4e0a\u5206\u522b\u4e3a VPG\u3001PPO\u3001A2C \u4e0e DQN \u5728 P100 GPU \u4e0a\u7684\u8bad\u7ec3\u7ed3\u679c\u3002\u53ef\u4ee5\u770b\u5230\uff0c\u6211\u4eec\u7684\u6d4b\u8bd5\u7ed3\u679c\u4e0e\u9879\u76ee\u63d0\u4f9b\u7684\u7ed3\u679c\u51fa\u5165\u4e0d\u5927\u3002<\/span><\/section>\n
      <\/section>\n
      \u7531\u4e8e CartPole \u4efb\u52a1\u5728\u5f3a\u5316\u5b66\u4e60\u4e2d\u76f8\u5bf9\u7b80\u5355\uff0c\u76f8\u5f53\u4e8e\u56fe\u50cf\u8bc6\u522b\u4e2d\u7684 MNIST\u3002\u4e3a\u66f4\u8fdb\u4e00\u6b65\u6d4b\u8bd5\u8be5 RL \u6846\u67b6\u7684\u6027\u80fd\uff0c\u6211\u4eec\u4e5f\u5728 MinitaurBulletEnv-v0 \u4efb\u52a1\u4e2d\u5bf9\u5176\u8fdb\u884c\u4e86\u6d4b\u8bd5\u3002<\/span><\/section>\n
      <\/section>\n
      Minitaur \u662f PyBullet \u73af\u5883\u4e2d\u4e00\u4e2a\u56db\u8db3\u673a\u5668\u4eba\u8fd0\u52a8\u63a7\u5236\u4efb\u52a1\uff0c\u5176\u89c2\u6d4b\u503c\u4e3a\u8be5\u673a\u5668\u4eba\u7684\u4f4d\u7f6e\u3001\u59ff\u6001\u7b49 28 \u4e2a\u72b6\u6001\u4fe1\u606f\uff0c\u63a7\u5236\u8f93\u5165\u4e3a\u7535\u673a\u7684\u8f6c\u77e9\uff08\u6bcf\u6761\u817f 2 \u4e2a\u7535\u673a\uff0c\u603b\u5171 8 \u4e2a\u7535\u673a\uff09\uff0c\u7b56\u7565\u4f18\u5316\u7684\u76ee\u6807\u4e3a\u6700\u5927\u5316\u673a\u5668\u4eba\u79fb\u52a8\u901f\u5ea6\u7684\u540c\u65f6\u6700\u5c0f\u5316\u80fd\u91cf\u6d88\u8017\u3002\u4e5f\u5c31\u662f\u8bf4\uff0cagent \u9700\u8981\u6839\u636e\u5956\u8d4f\u503c\u81ea\u4e3b\u5730\u5b66\u4e60\u5230\u7531 28 \u4e2a\u72b6\u6001\u4fe1\u606f\u5230 8 \u4e2a\u63a7\u5236\u8f93\u5165\u7684\u6620\u5c04\u5173\u7cfb\u3002<\/span><\/section>\n
      <\/section>\n
      \"\u6765\u81ea\u672c\u79d1\u751f\u7684\u66b4\u51fb\uff1a\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0\"<\/p>\n
      <\/section>\n
      \u4f7f\u7528 SAC \u7b97\u6cd5\u5728 Minitaur \u4efb\u52a1\u4e2d\u7684\u8bad\u7ec3\u7ed3\u679c\u5982\u4e0b\u56fe\u6240\u793a\uff1a<\/span><\/section>\n
      <\/section>\n
      \"\u6765\u81ea\u672c\u79d1\u751f\u7684\u66b4\u51fb\uff1a\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0\"<\/p>\n
      <\/section>\n
      \"\u6765\u81ea\u672c\u79d1\u751f\u7684\u66b4\u51fb\uff1a\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0\"<\/p>\n
      <\/section>\n
      \u9700\u8981\u6ce8\u610f\u7684\u662f\uff0c\u5929\u6388\u7684 SAC \u5b9e\u73b0\u5728 Minitaur \u4efb\u52a1\u4e2d\u4ec5\u8bad\u7ec3\u4e86\u4e0d\u5230 200k \u6b65\u5373\u80fd\u83b7\u5f97\u4ee5\u4e0a\u63a7\u5236\u7b56\u7565\uff0c\u6548\u679c\u53ef\u4ee5\u8bf4\u662f\u5f88\u4e0d\u9519\u7684\u3002<\/span><\/section>\n
      <\/section>\n
      \u9879\u76ee\u4f5c\u8005\uff0c\u6e05\u534e\u672c\u79d1\u751f<\/span><\/strong><\/span><\/section>\n
      <\/section>\n
      \u5728 GitHub \u4e2d\uff0c\u5176\u5c55\u793a\u4e86\u8be5\u9879\u76ee\u7684\u4e3b\u8981\u4f5c\u8005\u662f Jiayi Weng \u4e0e Minghao Zhang\uff0c\u4ed6\u4eec\u90fd\u662f\u6e05\u534e\u7684\u672c\u79d1\u751f\u3002\u5176\u4e2d Jiayi Weng \u4eca\u5e74 6 \u6708\u4efd\u672c\u79d1\u6bd5\u4e1a\uff0c\u5728\u6b64\u4e4b\u524d\u4f5c\u4e3a\u672c\u79d1\u7814\u7a76\u8005\u4e0e\u6e05\u534e\u5927\u5b66\u82cf\u822a\u3001\u6731\u519b\u7b49\u8001\u5e08\u5f00\u5c55\u5f3a\u5316\u5b66\u4e60\u9886\u57df\u7684\u76f8\u5173\u7814\u7a76\u3002Minghao Zhang \u76ee\u524d\u662f\u6e05\u534e\u5927\u5b66\u8f6f\u4ef6\u5b66\u9662\u7684\u672c\u79d1\u4e8c\u5e74\u7ea7\u5b66\u751f\uff0c\u540c\u65f6\u8fd8\u4fee\u4e86\u6570\u5b66\u4e13\u4e1a\u3002<\/span><\/section>\n
      <\/section>\n
      \"\u6765\u81ea\u672c\u79d1\u751f\u7684\u66b4\u51fb\uff1a\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0\"<\/p>\n
      <\/section>\n
      \u4f5c\u4e3a\u672c\u79d1\u751f\uff0c\u8be5\u9879\u76ee\u7684\u4e24\u4f4d\u4f5c\u8005\u5df2\u7ecf\u6709\u4e86\u975e\u5e38\u4e30\u5bcc\u7684\u7814\u7a76\u7ecf\u9a8c\uff0cJiayi Weng \u53bb\u5e74\u590f\u5b63\u5c31\u4f5c\u4e3a\u8bbf\u95ee\u5b66\u751f\u5230\u8bbf MILA \u5b9e\u9a8c\u5ba4\uff0c\u5e76\u4e0e Yoshua Bengio \u5f00\u5c55\u4e86\u5173\u4e8e\u610f\u8bc6\u5148\u9a8c\u76f8\u5173\u7684\u7814\u7a76\u3002\u5728 Jiayi Weng \u7684\u4e3b\u9875\u4e2d\uff0c\u6211\u4eec\u53ef\u4ee5\u770b\u5230\u5728\u672c\u79d1\u671f\u95f4\u5df2\u7ecf\u53d1\u4e86 IJCAI \u7684 Oral \u8bba\u6587\u3002<\/span><\/section>\n
      <\/section>\n
      \"\u6765\u81ea\u672c\u79d1\u751f\u7684\u66b4\u51fb\uff1a\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0\"<\/p>\n
      <\/section>\n
      Minghao Zhang \u4e5f\u6709\u4e30\u5bcc\u7684\u7814\u7a76\u7ecf\u9a8c\uff0c\u4e4b\u524d\u4ed6\u5728\u8f6f\u4ef6\u5b66\u9662 iMoon Lab \u505a\u5173\u4e8e 3D \u89c6\u89c9\u76f8\u5173\u7684\u7814\u7a76\uff0c\u800c\u540e\u76ee\u524d\u5728\u6e05\u534e\u4ea4\u53c9\u4fe1\u606f\u5b66\u9662\u505a\u7814\u7a76\u52a9\u7406\uff0c\u4ece\u4e8b\u5f3a\u5316\u5b66\u4e60\u65b9\u9762\u7684\u7814\u7a76\u3002\u5c3d\u7ba1\u79bb\u6bd5\u4e1a\u8fd8\u6709\u4e0d\u77ed\u7684\u65f6\u95f4\uff0cMinghao Zhang \u5df2\u7ecf\u505a\u51fa\u4e86\u81ea\u5df1\u7684\u7814\u7a76\u6210\u679c\u3002<\/span><\/section>\n
      <\/section>\n
      \"\u6765\u81ea\u672c\u79d1\u751f\u7684\u66b4\u51fb\uff1a\u6e05\u534e\u5f00\u6e90\u300c\u5929\u6388\u300d\u5f3a\u5316\u5b66\u4e60\u5e73\u53f0\uff0c\u7eafPyTorch\u5b9e\u73b0\"<\/p>\n
      <\/section>\n
      \u6240\u4ee5\u7efc\u5408\u6765\u770b\uff0c\u56e0\u4e3a\u5728\u672c\u79d1\u5df2\u7ecf\u6709\u4e86\u4e30\u5bcc\u7684\u79d1\u7814\u7ecf\u9a8c\uff0c\u5e76\u4e14\u505a\u8fc7\u591a\u4e2a\u9879\u76ee\uff0c\u90a3\u4e48\u5728\u8fd9\u4e2a\u9636\u6bb5\u80fd\u505a\u4e00\u4e2a\u975e\u5e38\u4e0d\u9519\u7684\u5f3a\u5316\u5b66\u4e60\u5f00\u6e90\u9879\u76ee\u4e5f\u5c31\u7406\u6240\u5f53\u7136\u4e86\u3002<\/span><\/section>\n
      <\/section>\n
      \u63a5\u4e0b\u6765\u7684\u5de5\u4f5c<\/span><\/strong><\/span><\/section>\n
      <\/section>\n
      \u5929\u6388\u76ee\u524d\u8fd8\u5904\u4e8e\u521d\u671f\u5f00\u53d1\u9636\u6bb5\uff0c\u5c1a\u6709\u4e00\u4e9b\u672a\u5b9e\u73b0\u7684\u529f\u80fd\u6216\u6709\u5f85\u5b8c\u5584\u7684\u5730\u65b9\u3002\u9879\u76ee\u4f5c\u8005\u8868\u793a\u4eca\u540e\u4e3b\u8981\u5728\u4ee5\u4e0b\u51e0\u4e2a\u65b9\u9762\u6765\u5b8c\u5584\u8be5 RL \u6846\u67b6\uff1a<\/span><\/section>\n
      <\/section>\n