If by 'directly', you mean coherent detection (instantaneous amplitude and phase), then direct measurements are possible using fast reference oscillators and heterodyne detectors for EM light up to the GHz range or so (millimeter waves). That may have been extended to the THz regime recently, but I have no reference for that.
It's possible to perform heterodyne measurements of visible light using an optical frequency standard as well, and I suppose the recent generation of an optical frequency comb could be used as an alternative, but it's not clear by your question what you mean by 'direct' measurement (i.e. counting cycles?)