The recent construction of integrable quantum field theories on two-dimensional Minkowski space by operator-algebraic methods is extended to models with a richer particle spectrum, including finitely many massive particle species transforming under a global gauge group. Starting from a two-particle S-matrix satisfying the usual requirements (unitarity, Yang-Baxter equation, Poincaré and gauge invariance, crossing symmetry, ...), a pair of relatively wedge-local quantum fields is constructed which determines the field net of the model. Although the verification of the modular nuclearity condition as a criterion for the existence of local fields is not carried out in this paper, arguments are presented that suggest it holds in typical examples such as nonlinear O(N) $\sigma$-models. It is also shown that for all models complying with this condition, the presented construction solves the inverse scattering problem by recovering the S-matrix from the model via Haag-Ruelle scattering theory, and a proof of asymptotic completeness is given.