A short paper, co-authored with Yutaka Matsuno is accepted by ICFEM 2020. Author version
Our paper “Falsification of Cyber-Physical Systems Using Deep Reinforcement Learning, Yoriyuki Yamagata ; Shuang Liu ; Takumi Akazaki ; Yihai Duan ; Jianye Hao”(Author version, Publisher Early Access) will be published in IEEE Transaction on Software Engineering (TSE). In this paper, we apply deep reinforcement learning to the falsification problem of CPS. This paper is the extended version FM2018 paper (ArXiv, Published version). We explained the theory more thoroughly and conducted three case studies with detailed statistical analyses.