The origin of ultra-compact dwarfs (UCDs; rh ≳ 10 pc) - objects larger and more massive than typical globular clusters (GCs), but more compact than typical dwarf galaxies - has been hotly debated in the 15 years since their discovery. Even whether UCDs should be considered galactic in origin, or simply the most extreme star clusters, is not yet settled. We present the dynamical properties of 97 spectroscopically confirmed UCDs and 911 GCs associated with the central cD galaxy of the Virgo cluster, M87. Our UCDs, of which 89% have M s ≳ 2× 106 MO and 92% are as blue as the classic blue GCs, nearly triple the confirmed sample of Virgo UCDs, providing by far the best opportunity for studying global dynamics of a UCD system. We found that (1) UCDs have a surface number density profile that is shallower than that of blue GCs in the inner 70 kpc and as steep as that of red GCs at larger radii; (2) UCDs exhibit a significantly stronger rotation than GCs, and blue GCs seem to have a velocity field that is more consistent with that of the surrounding dwarf ellipticals than with that of UCDs; (3) UCDs have an orbital anisotropy profile that is tangentially biased at radii ≲40 kpc and radially biased farther out, whereas blue GCs become more tangentially biased at larger radii beyond 40 kpc; (4) GCs with M s ≳ 2 × 106 MO have rotational properties indistinguishable from the less massive ones, suggesting that it is the size, instead of mass, that differentiates UCDs from GCs as kinematically distinct populations. We conclude that most UCDs in M87 are not consistent with being merely the most luminous and extended examples of otherwise normal GCs. The radially biased orbital structure of UCDs at large radii is in general agreement with the "tidally threshed dwarf galaxy" scenario.