A Planck 2015 CMB temperature individual pixel variance

[tjsotherone]

Hi,

I'm hoping someone can point me in the right direction please.

I'm using the Planck 2015 CMB temperature (intensity) SMICA pipeline maps (Nside = 2048) and am trying to determine the temperature variance of each individual pixel. Variance and hit-count were provided with the 2013 CMB maps (there were 3 columns: intensity, hit-count and variance) but they aren't in the 2015 maps (there is only an intensity column).

I've tried to estimate the variance from the half-difference of the half-mission and half-ring maps but my results look dodgy. I'm also very aware of the caution the Planck collaboration urge using these maps to estimate noise, and this approach doesn't feel well physically or statistically motivated to me.

There must have been hundreds of people using the Planck 2015 maps and I would imagine many of them would need to estimate temperature variance. But I can't find a standard/official approach in the literature or online. In fact, wherever I can find mention of variance it is usually rather vague and certainly not detailed enough for me to replicate the calculations.

Does anyone know whether the Planck collaboration have released the temperature variances? And if not, do you know if there is there a recommended method to derive them?

Many thanks,

Tracey.

 

[kimbyd]

They are available in the Planck Legacy Archive:
https://pla.esac.esa.int/pla/

Select "maps" then "advanced search". If you select the "exclude light maps" option, it will only show maps that have per-pixel errors attached. You're looking for maps with names like "LFI_SkyMap_070_2048_R2.01_full.fits". If you click the magnifying glass next to the map, it shows you what the columns are. The temperature variance is the II_cov column.

 

[tjsotherone]

Dear kimbyd,

Thank you so much for your reply. I hadn't used the advanced search so that's really useful!

My understanding is that those maps are from individual instruments and individual frequency channels. So the example you used presumably is from the low frequency instrument 70GHz channel. I assume (always dangerous) that the variance is only applicable for that specific map and not the one I'm using (COM_CMB_IQU-smica-field-Int_2048_R2.01_full.fits). Do you know whether that's the case please?

Maybe there is a way to combine all the individual variances into a single SMICA one but I haven't found it yet...

Many thanks,

Tracey.

 

[kimbyd]

Dear kimbyd,

Thank you so much for your reply. I hadn't used the advanced search so that's really useful!

My understanding is that those maps are from individual instruments and individual frequency channels. So the example you used presumably is from the low frequency instrument 70GHz channel. I assume (always dangerous) that the variance is only applicable for that specific map and not the one I'm using (COM_CMB_IQU-smica-field-Int_2048_R2.01_full.fits). Do you know whether that's the case please?

Maybe there is a way to combine all the individual variances into a single SMICA one but I haven't found it yet...

Many thanks,

Tracey.

There are a number of different ones. The one I pointed out is a map that is aggregated over the entire mission for all detectors in the 70GHz channel.

And yes, the variance is only applicable for the specific frequency map, not for the SMICA map, which is an aggregation over all of the maps.

Sadly, I'm not completely sure where or if they published per-pixel noise for the CMB maps. The component map products are described in detail here:
https://wiki.cosmos.esa.int/planckpla2015/index.php/CMB_and_astrophysical_component_maps

I don't see any mention of a noise map, but they do mention that you could use the sum/difference between the half-ring maps to produce a noise map estimate (the half-ring sum is in column HRHS, while the difference is in HRHD). Note the caveats they mention above.

 

[tjsotherone]

After much gnashing of teeth (and much help from the IRSA and PLA folks - thank you!) I have a couple of solutions that have worked for me (all I'm doing is weighting a few pixels). Here they are in case they help anyone else out:

(1) the II covariances in the 143 & 217 GHz maps give good enough estimates of noise for weighting.
(2) the HMHD maps do work if you smooth them (which I hadn't previously).

 

[Chronos]

While it may seem obvious, the Planck Legacy Archive does not offer raw instrument data as noted here https://commons.lbl.gov/download/attachments/111706420/DMP_planck.pdf:
"Plan for Serving Data to the Collaboration and Community:
Data are released to the public with the major Planck science publication releases. Both ESA and
NASA have dedicated archives to distribute the data, along with complete descriptions of their
contents. There is no formal plan to release the raw data processing pipelines since their use
depends on very detailed instrument knowledge that will not survive the collaboration."
For discussion of the processing steps traken to convert HFI raw instument data into published outputs, this may be of interest: http://planck.caltech.edu/pub/2015results/Planck_2015_Results_VII_HFI_Data_Proc_TOI_Beams.pdf

 

[kimbyd]

While it may seem obvious, the Planck Legacy Archive does not offer raw instrument data as noted here https://commons.lbl.gov/download/attachments/111706420/DMP_planck.pdf:
"Plan for Serving Data to the Collaboration and Community:
Data are released to the public with the major Planck science publication releases. Both ESA and
NASA have dedicated archives to distribute the data, along with complete descriptions of their
contents. There is no formal plan to release the raw data processing pipelines since their use
depends on very detailed instrument knowledge that will not survive the collaboration."
For discussion of the processing steps traken to convert HFI raw instument data into published outputs, this may be of interest: http://planck.caltech.edu/pub/2015results/Planck_2015_Results_VII_HFI_Data_Proc_TOI_Beams.pdf

Yeah, I don't think it would make sense for anybody else to examine the raw data. I was actually involved in some of that analysis a few years ago (on the LFI side), and the data analysis is quite complex. Some of the things you have to worry about:
1) The time constants of the detectors (once a detector passes a bright source, how long does it take for the detector to recover?).
2) The shapes of what is known as the beam (usually called the point spread function in optical astronomy, this is the shape of the blob that a point source appears to be when viewed by the detector). These are not known precisely until the satellite flies, so they have to be computed from observations of bright objects (e.g. planets). Each detector has a slightly different beam shape, and that shape changes depending upon which polarization signal you're looking at.
3) There was a 1Hz square wave that appeared in the LFI detectors that had to be removed.
4) The telescope will receive some amount of signal from locations far from the direction the detector is pointing. The satellite was designed to minimize this, but it still makes for a substantial contribution (this is known as the "far side lobes" of the beam.
5) The precise angles of the two polarization arms for each detector weren't known until the satellite flew, making it challenging to correctly estimate the polarization signal.
6) The refrigeration units slowly degraded over time, causing various temperature fluctuations which impacted the signal.

Anyway, if you're at all curious as to the work that was involved in making the Planck data useful, and some of the work that would have to be redone to make use of the raw data, you can take a look at these papers:
https://arxiv.org/abs/1507.08853 (LFI systematic uncertainties)
https://arxiv.org/abs/1605.02985 (HFI polarization systematics)
https://arxiv.org/abs/1502.01586 (HFI time-ordered data processing)