Interference effects in the decays of spin-zero resonances into γγ and tt¯

Abstract: We consider interference effects in the production via gluon fusion in LHC collisions at 13 TeV and decays into γγ and tt¯ final states of one or two putative new resonant states Φ, assumed here to be scalar and/or pseudoscalar particles. Although our approach is general, we use for our numerical analysis the example of the putative 750 GeV state for which a slight excess was observed in the initial LHC 13 TeV data. We revisit previous calculations of the interferences between the heavy-fermion loop-induced gg → Φ → γγ signal and the continuum gg → γγ QCD background, which can alter the production rate as well as modify the line-shape and apparent mass. We find a modest enhancement by ∼ 20% under favorable circumstances, for a large Φ width. The effect of interference on the apparent scalar-pseudoscalar mass difference in a two-Higgs-doublet model is found to be also modest. An exploratory study indicates that similar effects are to be expected in the gg → Φ → Zγ channel. In this and other models with a large Φ total width, the dominant Φ decays are expected to be into tt¯ final states. We therefore also study the effects of interference of the gg → Φ → tt¯ signal with the gg → tt¯ continuum QCD background and show that in the presence of standard fermions only in the gg → Φ loops, it is destructive causing a dip in the tt¯ mass distribution. Including additional vector-like quarks leads to a different picture as peaks followed by dips can then occur. We use the absence of such effects in ATLAS and CMS data to constrain models of the production and decays of the Φ state(s).