In this paper we present a rigid registration approach for 4D ultrasound (US) datasets, where images are registered over time. The 3D registration approach preceding the 4D registration consists of two main steps - block-matching and outlier rejection. The outlier rejection step removes the spurious matchings' from the block-matching module and ensures inverse consistency. For 4D registration, we perform registration of consecutive US volumes over the time series. Transformation between any two frames is estimated by taking the product of all the intermediate transforms. To avoid accumulation of error over the series of transformations, a long range feedback mechanism is proposed. A mean total registration error of 1 mm is achieved across six 4D ultrasound sequences of human liver with an execution speed of 10 Hz.