We study generalized electric/magnetic duality in Abelian gauge theory by combining techniques from locally covariant quantum field theory and Cheeger-Simons differential cohomology on the category of globally hyperbolic Lorentzian manifolds. Our approach generalizes previous treatments using the Hamiltonian formalism in a manifestly covariant way and without the assumption of compact Cauchy surfaces. We construct semi-classical configuration spaces and corresponding presymplectic Abelian groups of observables, which are quantized by the CCR-functor to the category of $C^*$-algebras. We demonstrate explicitly how duality is implemented as a natural isomorphism between quantum field theories. We apply this formalism to develop a fully covariant quantum theory of self-dual fields.