Interferometric visibilities, reflecting the complex correlations between signals recorded at antennas in an interferometric array, carry information about the angular structure of a distant source. While unknown antenna gains in both amplitude and phase can prevent direct interpretation of these measurements, certain combinations of visibilities called closure phases and closure amplitudes are independent of antenna gains and provide a convenient set of robust observables. However, these closure quantities have subtle noise properties and are generally both linearly and statistically dependent. These complications have obstructed the proper use of closure quantities in interferometric analysis, and they have obscured the relationship between analysis with closure quantities and other analysis techniques such as self calibration. We review the statistics of closure quantities, noting common pitfalls that arise when approaching low signal to noise due to the nonlinear propagation of statistical errors. We then develop a strategy for isolating and fitting to the independent degrees of freedom captured by the closure quantities through explicit construction of linearly independent sets of quantities along with their noise covariance in the Gaussian limit, valid for moderate signal to noise, and we demonstrate that model fits have biased posteriors when this covariance is ignored. Finally, we introduce a unified procedure for fitting to both closure information and partially calibrated visibilities, and we demonstrate both analytically and numerically the direct equivalence of inference based on closure quantities to that based on self calibration of complex visibilities with unconstrained antenna gains.