Precision Definition and 117 Threads

When I was first learning programming I was surprised that computers only hold numbers to 32 or 64 bits of precision (sometimes more). What I am wondering is to what precision does stuff in the universe happen? For example: if you had a super slow motion microscope and could zoom into a...

How do I construct a set of "concentric" circles with millimeter precision? I would need a few real life circular objects with the following specifications: They should all be "semi-concentric" so that they all share the same central axis (at right angle to the planes of the circles), but...

[SIZE="1"]This is a crosspost from General Math; I wasn't sure what place was more appropriate. I'm looking for a tool to solve a problem I've been working on. In particular, I want something that implements subquadratic trig functions -- in my case I'm looking for the tangent. I've been...

In my lab class, I needed to find k for a spring in two ways (hanging a mass from a spring & through simple harmonic motion). I've done this and now need to compare the two values - the question asks me to determine "the precision, expressed as a percent" of my measurements for k - I don't...

[SOLVED] Can someone check my precision for a meter stick, vernier cailper, and micro Meterstick Precision: 1 mm Example: 607 mm Vernier Caliper Precision: .1 mm Example: 41.5 mm Micrometer Precision : .01 mm Example: .05 mm what I'm asking basically is, is the precision I...

I've written a program that repeats a calculation a certain number of times in single precision, then in double precision. The times are recorded by the cpu_time(real t1) subroutine. (Specifically, the calculation is aReal/(aReal+0.01) .) It seems that the time for the double calculation is 1.20...

I am currently investigating ways to make a mold for hot-embossing. Normally this is done by for instance SU8 on a silicon wafer, or DRIE of a silicon wafer. But my design is deeper than a normal wafer, around 5 mm deep. Because the devices are actually fairly large (for microsystems) I...

Great, i get home from work and find my fairly expensive dell precision machine, which i use as a web server, in some state. All three fans spinning very loudly at full power like a jet engine and the computer completely dead. When i restart the computer no characters are printed, the monitor...

I'm trying to calculate some simple constants to high precision -- a few tens of millions of decimal places, maybe even a few hundred million. Are there any good programs out there that are fast at doing this? I'm using PARI/GP right now, but it's taking quadratic time to calculate ln(2)...

Hey.. I'm having trouble achieving this in C++... I'm trying to do an opperation but keeping only 2 decimals, and floor rounding. ((3.52 * 4.32)/3.26) + 2.34 So I have 3.52 * 4.32 = 15.2064, but then I just round to 15.20 then 15.20/3.26 = 4.662576687116, but then I just round to...

What level of precision is implied in an order-of-magnitude calculation?

help! with building a circuit. i need a gain of 300 (not exact, but somewhere around there, 200-500) for a small signal (typ. 0.1mV to 5mV) AC input with a bandwidth from 0.5Hz to 120Hz. I need it low-noise, with high precision and reliability... has anyone got a reference for any sort of help...

hi, how to determine the machine precision in MATLAB and estimate from this how many bits are used in the mantissa ??

Hi. I was wondering how do you calcultate physics problems when you must also note the error of measurement. I mean if you have a weight-scale that has a percison of 20 g, and let's say that the scale shows 0,7 kg. Does that mean you have 0,7 kg +- 0,02 kg or 0,7 +- 0,01 kg. Or what. And...

I'm trying to calculate the velocity of 10^20 eV cosmic rays, but the precision in Mathematica and Google is not high enough: both give velocities of exactly 1*c. I've tried putting SetAccuracy[...,1000] around all the constants and the final result, and it still doesn't work. It now...

I couldn't find a forum section on numerical analysis, so I'm writing this here. I'm on the lookout for simple matrix rotation/multiplication methods that can overcome the precision problems associated with poorly conditioned matrices. In my case I'm trying to simulate the rotational...

During the last few years I have read a number of books on The Big Bang including recently one By John Gibbin; "In the Beginning." Most if not all of the books mentioned the precision with which the initial values of some important parameters had to be for the universe to have formed as it has...