We propose a novel representation of cosmic microwave anisotropy maps, where each multipole order l is represented by l unit vectors pointing in directions on the sky and an overall magnitude. These "multipole vectors and scalars" transform as vectors under rotations. Like the usual spherical harmonics, multipole vectors form an irreducible representation of the proper rotation group SO(3). However, they are related to the familiar spherical harmonic coefficients, alm, in a nonlinear way, and are therefore sensitive to different aspects of the CMB anisotropy. Nevertheless, it is straightforward to determine the multipole vectors for a given CMB map and we present an algorithm to compute them. Using the WMAP full-sky maps, we perform several tests of the hypothesis that the CMB anisotropy is statistically isotropic and Gaussian random. We find that the result from comparing the oriented area of planes defined by these vectors between multipole pairs 2<=l1!=l2<=8 is inconsistent with the isotropic Gaussian hypothesis at the 99.4% level for the ILC map and at 98.9% level for the cleaned map of Tegmark et al. A particular correlation is suggested between the l=3 and l=8 multipoles, as well as several other pairs. This effect is entirely different from the now familiar planarity and alignment of the quadrupole and octupole: while the aforementioned is fairly unlikely, the multipole vectors indicate correlations not expected in Gaussian random skies that make them unusually likely. The result persists after accounting for pixel noise and after assuming a residual 10% dust contamination in the cleaned WMAP map. While the definitive analysis of these results will require more work, we hope that multipole vectors will become a valuable tool for various cosmological tests, in particular those of cosmic isotropy.