Context. The formation of massive (O-type) stars through the same accretion processes as low-mass stars is problematic, mainly because of the feedback massive stars provide to the environment, which halts the accretion. In order to constrain theoretical models of high-mass star formation, observational signatures of mass accretion in O-type forming stars are desirable. The high-mass star forming region NGC 7538 IRS1 (distance = 2.7 kpc) is an ideal target, because VLBI measurements of CH3OH masers recently identified a triple system of high-mass young stellar object (YSOs) in the region: IRS1a, IRS1b, and IRS1c. The first two YSOs seem to be surrounded by rotating disks. Aims: We want to characterize physical conditions and kinematics of circumstellar molecular gas around O-type young stars. Sub-arcsecond resolution observations of highly-excited lines from high-density tracers are useful, since these probe the hottest and densest gas, which presumably is close to O-type forming stars, i.e., in disks and the innermost portions of envelopes. Methods: Using the Karl Jansky Very Large Array (JVLA), we have mapped the hot and dense molecular gas in the hot core associated with NGC 7538 IRS1, with ~0.''2 angular resolution, in seven metastable (J = K) inversion transitions of ammonia (NH3): (J,K) = (6, 6), (7, 7), (9, 9), (10, 10), (12, 12), (13, 13), and (14, 14). These lines arise from energy levels between ~400 K and ~1950 K above the ground state, and are observed in absorption against the HC-HII region associated with NGC 7538 IRS1. The CH3OH JK = 132 - 131 and CH3CN (2-1) lines were also included in our spectral setup, but only the former was detected. We also obtained sensitive continuum maps at frequencies between 25 and 35 GHz. Results: For each transition, we produced resolved images of total intensity and velocity field, as well as position-velocity diagrams. The intensity maps show that the NH3 absorption follows the continuum emission closely. With a 500 AU linear resolution, we resolve the elongated north-south NH3 structure into two compact components: the main core and a southernmost component. Previous observations of the radio continuum with a 0.''08 (or 200 AU) resolution, resolved in turn the compact core in two (northern and southern) components. The velocity maps of the compact core show a clear velocity gradient in all lines, which is indicative of rotation. It is possible that the rotation is not in an accretion disk but in a (circumbinary) envelope, containing ~40 M⊙ (dynamical mass). The core hosts a binary system of massive YSOs, associated with the two (northern and southern) components of the radio continuum, which have a separation of about 500 AU and velocities around -59 km s-1 and -56.4 km s-1, respectively. The southernmost component, separated by 1000 AU and resolved in our NH3 maps (0.''2 beamsize) from the core, is associated with a third massive YSO, with a velocity around -60 km s-1. These features correspond to the triple system of high-mass YSOs IRS1a, IRS1b, and IRS1c. In addition, we derive rotational temperatures, NH3 column densities, H2 gas densities, and gas masses from the NH3 data. Surprisingly, measurements of the hyperfine structure show total optical depths of 10-26 even for these highly-excited lines, among the largest found so far in hot NH3 in high-mass star forming regions. From ratios of optical depths as well as rotational temperature diagrams of the observed ortho and para transitions, we derive a rotational temperature ~ 280 K, NH3 column densities ~ 1.4-2.5 × 1019 cm-2, total H2 volume densities ~ 3.5-6.2 × 1010 cm-3, and a total gas mass in the range of 19-34 M⊙ (the latter two assume [NH3]/[H2] = 10-7), for the main core. For the southern component, we derive a temperature of 110 K, a molecular density of ~ 0.7-2 × 1010 cm-3 and a gas mass in the range of 4-12 M⊙. Conclusions: We conclude that NGC 7538 IRS1 is the densest hot molecular core known to date, containing a rotating envelope which hosts a multiple system of high-mass YSOs, possibly surrounded by accretion disks. Future JVLA observations in the A-configuration are needed to resolve the binary system in the core and may allow the study of the gas kinematics in the accretion disks associated with individual binary members.
