Abstract： A novel algorithm for collision detection in robot operation training system based on octree of adaptive subdivision and encoding algorithm was proposed in this paper. Using this method, this structure can not only guarantee the real-time, and has the good topology. This octree structure adopts three times of the average triangle of the axis aligned bounding box to set the cell size. It is reasonable and effective compared with the radius of sphere. Based on the triangle size and the optimal cell size, by using the adaptive triangular mesh subdivision and the centroid distance, the collision in the space was detected. The experimental simulation data verify the realtime and effectiveness of the method.