What is Data analysis: Definition and 99 Discussions

Data analysis is a process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusions, and supporting decision-making. Data analysis has multiple facets and approaches, encompassing diverse techniques under a variety of names, and is used in different business, science, and social science domains. In today's business world, data analysis plays a role in making decisions more scientific and helping businesses operate more effectively.Data mining is a particular data analysis technique that focuses on statistical modeling and knowledge discovery for predictive rather than purely descriptive purposes, while business intelligence covers data analysis that relies heavily on aggregation, focusing mainly on business information. In statistical applications, data analysis can be divided into descriptive statistics, exploratory data analysis (EDA), and confirmatory data analysis (CDA). EDA focuses on discovering new features in the data while CDA focuses on confirming or falsifying existing hypotheses. Predictive analytics focuses on the application of statistical models for predictive forecasting or classification, while text analytics applies statistical, linguistic, and structural techniques to extract and classify information from textual sources, a species of unstructured data. All of the above are varieties of data analysis.Data integration is a precursor to data analysis, and data analysis is closely linked to data visualization and data dissemination.

View More On Wikipedia.org
Recent contents Top users
  1. fluidistic

    How can scientists trust closed source programs?

    I wonder how can scientists trust closed source programs/softwares. How can they be sure there aren't bugs that return a wrong output every now and then? Assuming they use some kind of extensive tests that figures out whether the program behaves as it should, how can they be sure that the...
  2. U

    Background rejection / data analysis

    Hi, How can one calculate background rejection from a background sample applying cuts ??
  3. Borg

    JavaScript 100,000 miles of driving visualized

    I thought that I would share some interesting visualizations that I have created from some personal driving data. I've recently been introduced to the wonderful world of the D3.js Javascript library. D3 stands for Data-Driven Documents is an extremely powerful and versatile data visualization...
  4. T

    Mathematical modeling in particle physics

    So my university offers two programs focused on particle physics. One is simply masters in nuclear/particle physics, the other is masters in mathematical modelling with focus on particle physics. I want to go into mathematical modelling and I'm choosing what I will focus on. I'm not really...
  5. D

    How to Waterproof a Strain Gage?

    Hello all, Our project needs to implement a force (strain) gage underwater for one of our tests. Our budget is pretty tight and we can't afford a wash-down strain gage that can support the forces that we will be operating at (~500 lbs). Does anybody know a way to hermetically seal a strain...
  6. C

    Underdetermined vs Overdetermined Systems

    I'm trying to create a model which is of the form y = (a0 + a1l)[b0+MΣm=1 bmcos(mx-αm)] [c0 + NΣn=1 cn cos(nz-βn)] In the above system, l,x and z are independent variables and y is the dependent variable. The a, b and c terms are the unknowns. To solve for these unknowns, I have two separate...
  7. Ben Mercado

    Engineering How to get into programming with a Physics/Engineering background

    Hi Everybody! I've got a job doing stress analysis as an engineer at an aerospace company. My best days are when I get to spend all day writing VBA codes/macros to automate stress analysis procedures or whatnot. I was thinking about getting a certificate in Data Analysis aka Data Science at...
  8. S

    Issue Finding the Mean Squared Error / Polyfit MATLAB

    Homework Statement Homework Equations Given above The Attempt at a Solution I used polyfit, but my mean swuare errors are way bigger than they should be- don't see what is wrong with my code! My code is ugly btw, my apologies. %Hw 7 clear all close all y3=[1960; 1965; 1970; 1975; 1980...
  9. brainpushups

    Prob/Stats Evolution of Data Analysis Techniques: A Journey through History

    Is anyone aware of any book(s) that presents the history of data analysis techniques? I'm most interested in how scientists during the enlightenment dealt with uncertainties and how techniques for dealing with uncertainty developed over time.
  10. A

    Advice Needed for Geophysical Data Analysis

    Good day, i need your advice on my latest career.I have B.Sc in Physics,M.Sc in Solid Earth Geophysics and now currently doing my Ph.D in non linear dynamics,investigating of chaos in geophysical data using correlation dimension and lyapunov exponent.I need your advice on how to be grounded in...
  11. N

    Mossbauer data analysis - folding

    Hi there, this looks like a great forum :-) I would need help with the folding procedure for raw data from mossbauer spectroscopy. Our software doesn't, for some reason, fold the data correctly, so we want to attempt a manual procedure. But how do we determine the correct sense/direction of...
  12. A

    Can I Accurately Determine Uncertainty for Data Points with Varying x Values?

    I have a lot of measurements of some quantity y as a function of x. All these data points are such that no y_i is taken at the same x_i. So I want to fit some kind of function to all these data point, but I want an uncertainty in the y_i's. Normally if I had say 10 y_i measured at the same x_i...
  13. D

    Data analysis - Comparing temperature data from multiple sources

    Data analysis -- Comparing temperature data from multiple sources Dear Peter, I am Engineering student from Japan.I have installed 8 sensors,4 at rural and 4 at urban area.Those sensors measures the temperature and other property in time series format.Now, I am using one month data and...
  14. J

    Windows SW for advanced data analysis

    I wrote this small Excel macro to gather data from vehicles database: Sub ImportCarQueryDB() Call JSON2Excel("2,3,4,8,12,14,20,21,26,27,28,29,31,32,33") ' Specify which fields to extract/import End Sub Sub JSON2Excel(ByVal fields As String) Filename$ = "C:\temp\fiat2.txt"...
  15. muraii

    Source for Autodidactic Learning of Data Analysis Techniques

    I'm looking for a little crowd-sourced shepherding along the road to a more robust grasp of analytical methods for data munging and summarizing. I completed a BA in mathematics but focused on the pure thread, to the degree undergraduates can be said to focus on anyone vein. I was not a...
  16. I

    Estimate Probability of Excel Data Analysis Results

    Not sure where to post this question, but here goes In excel I have calculated the average annual rain for the period 1990-2012 to be 50.2mm . How do you estimate the probability that this measured average (1990-2012) is consistent with the long term mean annual rain for the period 1948-2012...
  17. fluidistic

    Data analysis , I don't understand why this isn't a Gaussian nor a Ma

    Data "analysis", I don't understand why this isn't a Gaussian nor a Ma I have downloaded all the elo ratings of all active chess players in May of the FIDE and I have made an histogram. I have plotted the result on a graph rating vs number of people with this rating. I do not understand why...
  18. M

    Mathematics courses useful in engineering, data analysis and modeling

    Hello, I am currently pursuing my degree in Mechanical Engineering, and was wondering what math courses were good to take after the basic math sequence has been completed (Calculus 1-3, Differential Eq & Linear Algebra.). I have an interest in mathematical modeling and data analysis, as...
  19. P

    Topological Data Analysis - Persistent Homology

    Hi, I am not a mathematician, but I have noticed some recent papers on this seemingly new field, called Topological Data Analysis (see this relevant paper). I have had an overview of the applications and it seems that when you have data points that were sampled from some source (e.g. an...
  20. C

    Hot Dog Data Analysis: Exploring Sodium and Calorie Content by Type

    Homework Statement I'm doing a final project for my probability and statistics class that involves analyzing data on the sodium (mg/hot dog) and calories contained in each of 54 major hot dog brands. The hot dogs are classified by type: beef, poultry, and meat (mostly pork and beef). 20...
  21. L

    Data analysis: 2D Surface fitting from raw data

    Hey everybody I've got a question for the programming savvy. I'm generating data which can generally be described by.. f(x,y) = Ʃn(an * xn)*Ʃn(bn * yn) Basically what I'm looking at is a data set which shows a surface which looks like about 1/4th of a wonky bowl. My trouble is...
  22. B

    Data Analysis: Automating Distribution Comparisons from Daily Stats

    Hey guys, I have a question of how to go about answering a question. I am trying to decide what coding/database language to learn next (most likely SQL with some access thrown in), but have an overall question. I am looking for an application or way to crunch daily statistical updates, then...
  23. P

    Data analysis: error calculation

    Homework Statement A=0,078m l=2,27m relative error of A: 0,01 relative error of l: 0,005 What is the error of: arctan(A/l)?
  24. Z

    Fraunhofer Lines Essay/General Data Analysis

    Hi, I'm doing an essay on how accurately can a student measure fraunhofer lines in the sun? I've done the experiment, gotten good results, wrote about the equipment, the procedure etc... However I'm not sure what to do now, and I need to make this essay longer, and my data analysis is a...
  25. B

    Data Analysis career prospects

    Hi physics forums. I'm a 4th year student in Applied Physics. Originally I had no idea what I wanted to do with my education, or what useful skills I was learning. This year I've got a lab course and a computational physics course that have piqued my interest. I'm really enjoying...
  26. S

    Gravitational wave data analysis. More of Signal processing techniques

    I am using the matched filtering technique to extract the data from a heavy noise background in the process of detection of gravitational waves. I calculate the correlation between the experimental data and a theoretical template. I have been told that the maximum of the correlation function...
  27. A

    Data analysis software for hydrology research

    Hello there! This is my first time posting, but I'm a long time reader. I could not find a more appropriate sub-forum. I have just been hired to compile and organize data for an arctic hydrology research project in Fairbanks, AK. They are studying climate change and have recorded a few...
  28. S

    Ellipsometric data analysis software?

    Hi all , I am trying to learn modelling and data analysis for ellipsometric data for different materials (ψ,Δ) . Trying to find a evaluation version of available modelling softwares or any free software. Can anybody help me ?? Thanks! S
  29. U

    Back-testing Stock Selection - Data Analysis Help Needed

    Hey everyone! I'm a finance grad and am doing my first big project back-testing some stock selection methods. I have spent the last few weeks writing a big vba program to run the back-test and I have the following: 10 dates (5 years semi-annual) and 40 companies where, for a given...
  30. D

    Mass spectrometry data analysis

    1. A liquid compound gave a mass spectrum showing a strong molecular ion at m/z = 156. The only fragment ions are seen at m/z = 127 and 29. Suggest a structure of this compound. 2. A liquid compound gave a mass spectrum in which the molecular ion appears as a pair of equal intensity peaks at...
  31. H

    Graphing and Data Analysis software?

    Hello everyone, I was wondering what software you all find useful for analyzing data. What I'm looking for needs to have extensive curve fitting abilities as well as error analysis tools, i.e., error bars and what not. I hope this isn't totally vague and you all have a piece of software in...
  32. L

    Raman spectroscopy: data analysis: convolution

    hey guys, i hope you can help. my task is to analyse data of raman spectroscopy. therefor i have to deconvolute it. that means the data must have been convoluted somewhere. is it true that the raw data which i receive is convoluted already? or is it common to convolute the data "active"...
  33. R

    Sample Calculation on Milikan's Data Analysis

    Homework Statement Find the charge on the oil drop in Coulombs, and the number of electrons it is "missing". Vplates= 2400 V Dplates = 0.020 m Oil Density = 850 kg/m3 Oil Drop Radius = .000051 m (from Stokes’ Law) Homework Equations E = V/d Volume = 4/3πr3 m= density x volume...
  34. J

    Fortran Data analysis with fortran/c or simulink

    Hello, Lately i had my XPS measurements from my collegue. data files are .txt files and contains two column and thousends of rows of consequent measurements of same region. i mean 81.510 aa.bbb ... ... 91.870 cc.ddd 81.510 ee.fff ... ... 91.870 gg.hhh the reason of...
  35. M

    John A. Rice book (Mathematical Statistics and Data Analysis)

    I am using this book for a mathematical statistics class I am in. I have an account with cramster to check answers since they have solutions to nearly the entire text, but they are HORRIBLE. Half of the time they are dead wrong and they always lack decent explanation which is pretty key in...
  36. E

    Which is better for data analysis: MatLab or Visual Basic?

    Hey there. I've this question in mind whether to pick between MatLab of Visual Basic to write my program for data analysis for Inertial Measurement Unit(IMU) in the ground system. May i know which is better and why? I learned c++ before only and currently trying to venture into the other...
  37. T

    Optimal Approach for Analyzing Non-Smooth Experimental Data

    Hi, I need your help. From experiments I got data set which I need to analyze. The problem is that my data is not smooth. I tried to fit my data using a polynomial equation, but the fitting was not good enough. I also tried to smooth, spline... but got very different final results. Can anyone...
  38. Q

    Programs PhD Scholar Leave and Research: Physics Theories and Data Analysis

    How much leave do phd scholars in physics get annualy? Is it it not possible to do research work at home,incase the student is ill and wants to be in his comfort zone during bad times? Btw if i choose a theoretical physics topic will i be free from learning computer related stuff with...
  39. S

    Fractional uncertainties (Data analysis)

    I've been trying to determine the Boltzmann constant by observation of Brownian motion. I undertook four experiments and hence got four different estimated values for kB. To analyse the data, I estimated the mean value and the standard deviation. This is maybe not the best way to analyze the...
  40. D

    Data Analysis: Gamma ray attenuation

    Hello, I'm attempting to analyse the data recovered from an experiment that I performed in lab, but I'm having some problems understanding how to properly apply the statistical methods learned to this specific problem. Essentially, the experiment consisted of placing a source of gamma rays...
  41. M

    Which method is more accurate for determining peak time in data analysis?

    Homework Statement Say I perform an experiment, and I make a number measurements over a given interval (e.g t=0s to t = 10s, every 1s), and I perform this experiment many times. Now, let's say I make a plot of data vs. time, and I want to find when the data peaks in time on average...
  42. Pythagorean

    Understanding the Form Factor and its Role in Rutherford Scattering

    Homework Statement This is for a lab class, I'm writing a report and giving a presentation. Tomorrow is the day and I've just received the final remarks on my lab writeup, most of which are simple and obvious enough, but this one really bugs me: I think my teacher is wrong. Rutherford's...
  43. N

    Simple Harmonic Motion Lab Data Analysis

    Here is the problem I am having: Professor told me to find the spring constant using our slope of the graph. Now the graph that I did on the excel went something like this: On the X-axis of the graph was the weight of the hanging mass in Newtons and on the Y-axis of the graph was the...
  44. L

    Understanding Non-Linear Data: Analyzing Quiz Scores for a Class of 18 Students

    I have trouble figuring out a data problem. Please take your time to help me out. Thanks in advance! 1. The ordered pairs represent the scores on two consecutive 15 point quizzes for a class of 18 students. (data provided) Why is the graph not linear? My solution: They provided me data and...
  45. S

    Solving Cow Industrials Ltd's Production Problem for Board of Directors

    Looking for guidance on the following: The text outlines a company's problem, and ways to solve it. Results are hypothetically to be presented to the 'Board of Directors' ina week's time. Help please! Cow Industrials Ltd is a medium sized Engineering Company. A number of their regular customers...
  46. S

    Data Analysis Fun: Helping Cow Industrials Ltd Reduce Absenteeism

    Looking for guidance on the following: The text outlines a company's problem, and ways to solve it. Results are hypothetically to be presented to the 'Board of Directors' ina week's time. Help please! Hope you can help meee :)
  47. A

    Investigating Time Taken by Ball Bearing Rolling Down a Slope

    hi chaps, I am doing this work on "investigating how the distance traveled by a ball bearing rolling down a slope affects the time taken" (i know riveting! lol) and what I've got so far is a table of results (distance it rolled and then how long it took) ive also got : angle of...
  48. S

    Data Analysis web site for experiments ?

    Is there a site where I could see what people have done in various experiments? For example, I will be doing the verlocity of light experiment and would like to know what kinds of data analysis people have done so I can get some ideas as to what I could do with my data. Is there any such...
  49. T

    Fig.5: Is the WMAP data analysis flawed?

    Note: if you want to avoid opening all the image links below separately, you can go to my webpage http://www.physicsmyths.org.uk/wmap.htm which is essentially identical to this post The Wilkinson Microwave Anisotropy Probe (WMAP) is a satellite carrying a twin radio telescope for measuring...
Back
Top