We provided a vision-controlled robotics manipulation system with a robust, accurate algorithm to predict the translational motion of a 3-D object; hence, making it possible to continuously point the video camera at the moving object. The real time video images are fed to a PVM-1 (a pyramid-based image processor) for image processing and moving object detection. The measured object coordinates are continuously fed to our algorithm for track smoothing and prediction. In this study, we examined several tracking algorithms and adopted an optimal α - β filter for tracking purposes and the α - β -γ filter as part of the initialization procedure. The optimum gains for these 6lkm are based on the Tracking Index principle which in its turn is based on the measurement noise variance and the object dynamics. We derived an expression for the noise variance corresponding to our application. As for the object dynamics, we developed an adaptive method (using the α - β -γ filter mentioned above) for inferring object dynamics in an iterative learning process that results in an accurate estimate of the Tracking Index. The accuracy of our algorithm realizes that of the Kalman filter but is much simpler computationally.