A robotic system can be characterized by its interactions with environments. With growing demand for robots deployed in various scenarios, the ability to perform physical interaction in uncontrolled environments has become of great interest. While a robot performs interactive tasks, its visual and spatial sensing plays a critical role. Being a major source of learning, vision not only guides immediate actions, but also indirectly improves future actions and decisions. How visual information is gathered and represented will significantly influence how a robot can plan and act. Although recent advances in machine perception have presented unprecedented performance in some areas, there still exist challenges in various aspects. In this dissertation, I will address two such issues and suggest an online probabilistic approach to each problem. Most successful approaches in visual learning depend on fragments of exemplars prepared by humans. It is simply unaffordable to provide constant human supervision to a robotic system that would receive tens of new image frames per second. Ideally, a robotic system is required to gather information from its unique experience and keep growing knowledge on the fly without such external aids. One way to implement the self-learning is to take advantage of the naturally correlated sensations of different sensory modalities. The first part of this talk presents a probabilistic online self-learning framework to alleviate the dependency in robotic visual learning by leveraging structural priors. Another challenge in robotics is its spatial understanding. Aside from planning and performing actions, spatial representation itself still largely requires more research. While point or grid-based representations are currently being employed for practical conveniences, these methods suffer from discretization and disconnected spatial information. On the other hand, Gaussian Processes (GP) have recently gained attention as an alternative to represent the distance field of structures continuously and probabilistically. It is not only the seamless expression of structures, but also direct access to the distance and direction to obstacles that make the representation invaluable. The second part of the talk presents an online framework for continuous spatial mapping using GP.