This paper describes a neuromorphic implementation of the orientation hypercolumns found in the mammalian primary visual cortex. A hypercolumn contains a group of neurons that respond to the same retinal location, but with different orientation preferences. The system consists of a single silicon retina feeding multiple orientation selective chips, each of which contains neurons tuned to the same orientation, but with different receptive field centers and spatial phases. All chips operate in continuous time, and communicate with each other using spikes transmitted by the asynchronous digital Address Event Representation communication protocol. This enables us to implement recurrent interactions between neurons within one hypercolumn, even though they are located on different chips. We demonstrate this by measuring shifts in orientation selectivity due to changes in the feedback.