In the future, an Exobyte of astronomical data per day

[rhody]

Wow, what a Project: Square Kilometer Array

Over the next 12 years, thousands of antennas will be built and installed across a 5000-kilometer stretch of the southern hemisphere. Satellite dishes, tripod-like dipole antennas, and tiled circular stations will dot arid savannas and comprise the world’s biggest, most accurate radio telescope ever constructed: the Square Kilometer Array.

The ambitious project, which brings 67 scientific teams from 20 countries together, is the next big thing in global scientific collaboration. (To clarify, the antennas cover continent-wide distances, but it’s the signal-collecting area that is one square kilometer, the equivalent of a single dish with a square kilometer of surface area.) Like CERN’s Large Hadron Collider (LHC), the SKA is a multi-year, multi-billion dollar enterprise aimed at answering some of the most fundamental questions about deep time and the very nature of the universe. According to Ronald Luijten, a senior manager at IBM’s Zurich Research Lab, “SKA is very similar to the CERN project in terms of the complexity of project itself, the size of the scientific community, and the global nature of the operation.”

Despite these structural and cultural similarities, the SKA represents a new step in terms of data management and the complexities of project coordination. The instrument will generate an exabyte of data every day – that would be 1,000,000,000,000,000,000 bytes – more than twice the information sent around the internet on a daily basis and 100 times more information than the LHC produces.

This thread is unique, last one was written in 2007 concerning the "Square Kilometer Array".
It looks like requirements will push new developments in hardware and software technologies to handle an exobyte of data per day. One half an exobyte of data sent around the internet today, March 2012. Someone, please mark and remember this remark to be revisited a decade from now.

Rhody...

 

[Chalnoth]

Wow, what a Project: Square Kilometer Array

This thread is unique, last one was written in 2007 concerning the "Square Kilometer Array".
It looks like requirements will push new developments in hardware and software technologies to handle an exobyte of data per day. One half an exobyte of data sent around the internet today, March 2012. Someone, please mark and remember this remark to be revisited a decade from now.

Rhody...

Yes, the growth of retrieval of astronomical data has been absolutely tremendous. Doubling ever year.

That means that every year, we obtain as much new astronomical data as has been collected in the entire history of human civilization.

 

[Drakkith]

Is the amount of data gathered so large because there are a great number of small telescopes as opposed to one big one?

 

[Crake]

I heard about this project today for the first time. It sounds really amazing. Hopefully it doesnt' get canceled!

 

[Chalnoth]

Is the amount of data gathered so large because there are a great number of small telescopes as opposed to one big one?

Heh, guess I never answered this post, though I should have.

It's a combination of things. Individual telescopes are getting higher-resolution camera elements which are more sensitive and thus can collect data at a higher rate. More telescopes are being built (such as the SKA that Crake posted above). New techniques such as adaptive optics (which uses a flexible mirror and a laser bright laser shined upward to create a false star in the upper atmosphere) correct for optical distortion from our atmosphere, which allows higher resolution as well. Larger telescopes are being built, such as the Thirty Meter Telescope: http://www.tmt.org/ (larger telescopes have a larger collecting area which allows us to collect the same image with less exposure time, or, if well-designed, probe the sky at higher resolutions). We're also probing the sky at new electromagnetic frequencies that we've never probed before, such as into the Gamma ray range with Fermi.