The recent discovery of the ?-ray emission from the lobes of the closest radio galaxy Centaurus A by Fermi implies the presence of high-energy electrons at least up to ? ~ 105-106. These high-energy electrons are required to interpret the observed ?-ray radiation in terms of inverse Compton emission off the cosmic microwave background (IC/CMB), the widely accepted scenario to describe the X-ray emission of radio galaxy lobes. In this Letter, we consider the giant radio lobes of FR II radio galaxies showing that it is possible to maintain electrons at energies ? ~ 105-106, assuming an acceleration scenario (driven by turbulent magnetic fields) that compensates radiative losses. In addition, we consider the contribution to the diffuse extragalactic ?-ray background due to the IC/CMB emission of FR IIs' lobes, showing its relevance in the keV to MeV energy range.