We present a general formulation for inverting frequency- or time-domain electromagnetic data using an all-at-once approach. In this methodology, the forward modeling equations are incorporated as constraints and, thus, we need to solve a constrained optimization problem where the parameters are the electromagnetic fields, the conductivity model, and a set of Lagrange multipliers. This leads to a much larger problem than the traditional unconstrained formulation where only the conductivities are sought. Nevertheless, experience shows that the constrained problem can be solved faster than the unconstrained one. The primary reasons are that the forward problem does not have to be solved exactly until the very end of the optimization process, and that permitting the fields to be away from their constrained values in the initial stages introduces flexibility so that a stationary point of the objective function is found more quickly. In this paper, we outline the all-at-once approach and apply it to electromagnetic problems in both frequency and time domains. This is facilitated by a unified representation for forward modeling for these two types of data. The optimization problem is solved by finding a stationary point of the Lagrangian. Numerically, this leads to a nonlinear system that is solved iteratively using a Gauss-Newton strategy. At each iteration, a large, indefinite matrix is inverted, and we discuss how this can be accomplished. As a test, we invert frequency-domain synthetic data from a grounded electrode system that emulates a field CSAMT survey. For the time domain, we invert borehole data obtained from a current loop on the surface.