In this paper I sketch a genealogy of racial regulation of labor mobility within empire and theorize the entanglement between transnational imperial discourses and popular (working class) politics in the West. By juxtaposing diversely located thinkers that sought to understand or influence the imperial politics of mobility and working class politics, I argue that the question of imperial mobility shaped emergent notions of popular sovereignty and tied them to racial immigration control. While white working classes throughout the British empire and the metropole sought to oppose capitalist tactics of labor control, they did so while embracing imperial racial discourse. As a result, popular sovereignty emerged as a claim to free mobility, self government, and enfranchisement for white European subjects, which comprised the denial of those rights to workers of color. This meaning of popular sovereignty is placed more radical contestations of the racial and transnational character of capitalism. Bringing to the forefront the imperial genealogy of immigration regulations disrupts liberal political theory frameworks that consider these restrictions legitimate attributes of sovereignty.