Recently, there has been a growing interest in the use of mosaic images to represent the information contained in video sequences. This paper systematically investigates how to go beyond thinking of the mosaic simply as a visualization device, but rather as a basis for an efficient and complete representation of video sequences. We describe two different types of mosaics called the static and the dynamic mosaics that are suitable for different needs and scenarios. These two types of mosaics are unified and generalized in a mosaic representation called the temporal pyramid. To handle sequences containing large variations in image resolution, we develop a multiresolution mosaic. We discuss a series of increasingly complex alignment transformation (ranging from 2D to 3D and layers) for making the mosaics. We describe techniques for the basic elements of the mosaic construction process, namely sequence {\em alignment}, sequence integration into a mosaic image, and residual analysis to represent information not captured by the mosaic image. We describe several powerful video applications of mosaic representations including video compression, video enhancement, enhanced visualization, and other applications in video indexing, search, and manipulation.