Big Data and RDBS (Relational DB). Do They Fit?

Click For Summary

Discussion Overview

The discussion revolves around the compatibility of Relational Databases (RDBS) with big data applications. Participants explore the challenges posed by the rapid inflow of data and the role of data warehouses in addressing these challenges. The conversation includes inquiries about the types of databases utilized by data warehouses and the relationship between corporate databases and data warehouses.

Discussion Character

  • Exploratory
  • Debate/contested
  • Technical explanation

Main Points Raised

  • Some participants express difficulty in understanding how RDBS can handle the fast inflow of data associated with big data.
  • There is a suggestion that data warehouses are intended to address the limitations of RDBS in big data contexts.
  • Participants mention NoSQL databases, such as MongoDB and Hadoop, as alternatives that may be used in big data scenarios.
  • One participant describes the function of data warehouses as extracting and organizing data from corporate databases for analysis and forecasting, focusing on product trends rather than customer behavior.
  • Another participant notes that RDBMS have improved in speed, reliability, and security, but emphasizes that a combination of RDBS and NoSQL may be necessary to balance workloads effectively.

Areas of Agreement / Disagreement

Participants do not reach a consensus on the effectiveness of RDBS in big data applications, with multiple viewpoints on the role of data warehouses and the use of NoSQL databases. The discussion remains unresolved regarding the best approach to integrating RDBS with big data.

Contextual Notes

Some assumptions about the capabilities of RDBS and NoSQL databases are present, but these are not fully explored or agreed upon. The discussion also reflects varying interpretations of how data warehouses function in relation to corporate databases.

WWGD
Science Advisor
Homework Helper
Messages
7,806
Reaction score
13,120
Hi All,
I am having trouble seeing how Relational Databases (RDBS) can be used in the world of big data.
The inflow of data seems to be way too fast for the database to reflect what is going on at a given
moment. I understand this issue is supposed to be addressed by data warehouses. Is this correct?
If so, what kind of databases are used by data warehouses?
 
Technology news on Phys.org
WWGD said:
Hi All,
I am having trouble seeing how Relational Databases (RDBS) can be used in the world of big data.
The inflow of data seems to be way too fast for the database to reflect what is going on at a given
moment. I understand this issue is supposed to be addressed by data warehouses. Is this correct?
If so, what kind of databases are used by data warehouses?
NoSQL databases like MongoDB, Hadoop, etc.
 
  • Like
Likes   Reactions: WWGD
Data warehouses use the same kinds of databases as "big data". Usually, information from the corporate database is extracted and organized into smaller data warehouses useful for doing analysis and forecasting of trends.

As an example, the corporate database (amazon) might contain information on customers, their purchases and wishlists, what they looked at... whereas the datawarehouse might not care about the customers buying behavior but only the products that were purchased when, where and how but not who and from that forecast what stores need what inventory.
 
  • Like
Likes   Reactions: WWGD
jedishrfu said:
Data warehouses use the same kinds of databases as "big data". Usually, information from the corporate database is extracted and organized into smaller data warehouses useful for doing analysis and forecasting of trends.

As an example, the corporate database (amazon) might contain information on customers, their purchases and wishlists, what they looked at... whereas the datawarehouse might not care about the customers buying behavior but only the products that were purchased when, where and how but not who and from that forecast what stores need what inventory.
Thanks. So it seems like a division of labor where corporate finds patterns and the warehouses deal with day to day?
 
RDBMS have themselves become more fast, reliable and secure - you can take examples of big vendors like Oracle, but there is no silver bullet in this case too, so as pointed out by jedishrfu some well - balanced combinations of both RDBS and NoSQL are used in practice, in order to have the most balanced workload. The exact "how" varies according to company's needs.
 
  • Like
Likes   Reactions: WWGD

Similar threads

  • · Replies 11 ·
Replies
11
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 5 ·
Replies
5
Views
1K
  • · Replies 20 ·
Replies
20
Views
3K
  • · Replies 5 ·
Replies
5
Views
4K
  • · Replies 2 ·
Replies
2
Views
6K
Replies
28
Views
4K
  • · Replies 10 ·
Replies
10
Views
2K
  • · Replies 9 ·
Replies
9
Views
4K
  • · Replies 6 ·
Replies
6
Views
6K