Calculating Heat Exchange: A Z80 Program with Real Weather Data

  • Thread starter anorlunda
  • Start date
In summary, a man lost millions while day trading oil when the price went negative due to a fault in the handling and display of quantitative values in his desktop software. This issue highlights the need for foolproof methods of displaying and handling quantitative values and the importance of proper requirements in software development. While similar to the Y2K problem in terms of faulty handling of numerical values, it differs in that the requirement for a negative price was unknown and not addressed properly. Overall, it serves as a cautionary tale for the potential consequences of overlooking important details in software development.
  • #1
anorlunda
Staff Emeritus
Insights Author
11,308
8,730
In this story, a man was day trading oil. On that day, the price went negative but his desktop software could not display a price lower than $0.01, so he lost millions.

https://www.bloomberg.com/news/arti...puters-and-inflicted-huge-losses?srnd=premium

How is this like Y2K? Just that it is a fault in the handling and display of quantitative values. I'm confident that the future will expose still more ways to get caught in this trap.

It is remarkable that with our great technology, we have not found a foolproof way of eliminating analogous problems in the future. A method to display quantitative values that can not be misinterpreted or mishandled.
 
  • Like
  • Wow
Likes ProfuselyQuarky, russ_watters, Drakkith and 2 others
Computer science news on Phys.org
  • #2
This is really unlike the Y2K problem. Computer programmers (the term "Software Engineer" was uncommon in those days) knew that the year 2000 was coming. But data storage (computer memory, tape drives, disk drives, etc) was tight. It was a well-known issue - but a lot of programmers just didn't put in the extra effort to keep their code working past 2000 - even code written well into the 90's.

Incidentally, I started coding professionally in 1971 and never wrote any code that would crash in 2000. In the simplest case, you just made sure that the 2-digit year was put to the best use. For example, code written in 1971 would need a way on dealing with birth dates in the range of 1870 to the codes end of life (say 1991). That's 121 years so you can't go with 2 digits. But if it's a date stamp created during the codes life created in 1971, you can interpret "69" as "2069".

This problem seems different in many ways. Unlike Y2K, it seems as though the requirement for a negative price was unknown. Negative numbers were turned into 0.01 - apparently as a method of defensive coding. Although, that type of "correction" is really risky. If you see a negative number and you believe that it is out of range, clipping it to stay within range will keep you from dividing by zero, but it is clearly not sufficient when used in a product that can cost it's users millions. If the price was really wrong, it should have resulted in more obvious actions by the software - such as a notice to the user that no valid information was available. If I had thought that negative numbers were not allowed, I would have filled the price value with a red "Missing Data" or "Data Error".

So Y2K were a design issues. This is much more of a requirements issue.
 
  • Like
Likes Klystron
  • #3
The Bloomberg article tends to support this idea.
.Scott said:
{snip...}
So Y2K were a design issues. This is much more of a requirements issue.
The person in the article trading oil from home, expected accurate price quotes. As a software engineer (:cool:) I would never trust a single numerical price readout to initiate trades, especially long or midterm 'buy & holds'. While I agree the software should seamlessly handle negative values or trip alarms and send error alerts, a canny trader should examine several sources for prices before committing capital.

Just for example this free site provides lists, charts, and graphs of a wide variety of stocks, bonds, funds, and commodities in real-time, post-time, as futures and derivatives; plus a wealth of common technical indicators such as Bollinger bands, moving (uh, gliding) averages, etc. A glance at a chart of crude oil prices would indicate the sharp price corrections in the petroleum markets that day/week/month. Most online trading companies provide similar software packages with free training and free advice to examine several sources before initiating trades.

Savvy traders also watch financial news alongside technical charts and screens. A glance at MSNBC or Bloomberg TV or Asian/Indian news sources should have twigged the trader to modify their buy strategy. Professional commodity traders probably appreciate uninformed 'buy' orders as oil prices tank.

Apologies if off topic from Y2K, but this story seems like programmer design errors exacerbated by user mistakes and inexperience. The correct price and petroleum market trend information was readily available. Concur that unexpected fluctuations in commodity prices into negative territory might be analogous to expected damage to computer systems from legacy systems that only provided two digits for year data as we entered the 21st Century.
 
  • #4
.Scott said:
It was a well-known issue - but a lot of programmers just didn't put in the extra effort to keep their code working past 2000 - even code written well into the 90's.
...
So Y2K were a design issues.
There were a great many systems where programmers were not able to address this issue unless it was made an official requirement. Programmers could not ok the funding, schedule adjustment, and risk acceptance that fixing the Y2K problem would require. This was a program management responsibility. I am sure that if it was an official requirement, most programmers would have worked on it much earlier.
 
  • #5
The very first pro program I wrote was for a mailing company, just after the country had implemented a postal code system. I did the postal code wrong : put six characters in a row, instead of the two groups of three required by the post office to handle bulk mail ; the client was dumbfounded that I didn't - magically in my eyes - know that. That was an easy issue to spot and fix.

I got to admit, were I the guy doing the stock software, I would assume that stocks don't have negative prices, and it might not occur to me to check (might). Of course that point's moot for two mutually exclusive reasons :

a) business analysts : they exist so companies don't get software with only a vague idea of how the business operates ;

b) anything considered an invalid input gets flagged and acted upon.

Relationship to Y2K ? Not really, at least not in terms of large mainframe shops : everybody knew there was upcoming drudgery : now, they may have spent millions updating the programs, but bear in mind they had saved tens of millions in data storage, transfer, and CPU cycles, over the previous years.
 
  • #6
anorlunda said:
In this story, a man was day trading oil. On that day, the price went negative but his desktop software could not display a price lower than $0.01, so he lost millions.
How is it possible to design software dealing with quantities and somehow ignore negative numbers? Since you are dealing with possibly large numbers, you routinely use FLOAT (or double in C terminology). The only case I would use unsigned would be in "counting" variables.
 
  • #7
Svein said:
How is it possible to design software dealing with quantities and somehow ignore negative numbers? Since you are dealing with possibly large numbers, you routinely use FLOAT (or double in C terminology). The only case I would use unsigned would be in "counting" variables.
My guess is that it is not the internal representation of the number, but rather its display on the screen. There might not be enough room in the text box for price to include the sign. For example, if I had only 5 characters, I could show 00.00 leaving no room for + or -. With extra effort, I might cleverly display other cases in 5 characters, but the programmers clearly didn't take the extra effort.

That is why in post #1 I said that a foolproof way of displaying numbers to humans in all cases remains an unsolved problem.
 
  • Like
Likes Klystron
  • #8
Y2K as a bug had been overblown by the press. The year 2000 Millennium was discussed interminably, at least since the last millennium. There were (are) several millenarian religions with sects that argue the precise date and time of the Change. Though I had studied theology and history, I was never certain how the thousand years was actually calibrated not what events were expected to occur.

I only mention religions since ecstatic millennarian myths and legends coloured public perception of an obvious design problem that had largely been mitigated by the 1990's. Programmers were always aware society used 4* integer digits to represent years. Two digit years represented frugal resource management in early years of computation and data collection.

This thread cites petroleum prices, not stocks or other financial instruments. Though outside my areas of expertise, I understand from simple supply vs. demand graphs that when supply exceeds demand that prices tend to fall in a market correction.

In the case of crude petroleum when demand falls -- in this case due to pandemic travel restrictions -- and the cost per barrel to extract the crude exceeds market price in a supply glut, then quoted prices become negative. In other words it costs more to produce a barrel of crude oil than the market is willing to pay due to 1) oversupply and 2) weak demand.

Well designed software meant to quantify and display commodity prices must be able to handle negative price corrections.

------------------------------------
*Some software, and some societies, represent years differently; usually from some accepted arbitrary start date. Examples include founding year of a city, the birth or death of some notable person or imaginary being, or the arrival of a tribe in a new location.
 
Last edited:
  • #9
Svein said:
How is it possible to design software dealing with quantities and somehow ignore negative numbers?
One almost always has to check the validity of user inputs to take care of typos and other user mistakes. It is very possible that there was a check programmed into the interface that would reject a user input of a negative cost.
Klystron said:
Y2K as a bug had been overblown by the press.
It was overblown, but it was still a serious problem. I spent several days changing the modification date of files on backup tapes so that files could be retrieved. It was a program requirement to have usable backups of old versions of code.
 
  • #10
Svein said:
routinely use FLOAT

In financial software using float is grounds for summary execution. Rounding errors could shift/create a fraction of a cent into the wrong account. I'm sure someone would find that and manufacture money by shifting the right fractions around.

BoB
 
  • #11
rbelli1 said:
In financial software using float is grounds for summary execution. Rounding errors could shift/create a fraction of a cent into the wrong account. I'm sure someone would find that and manufacture money by shifting the right fractions around.
But when calculating gain/loss of stock transactions, who cares about fractions of a cent when the amount is in the millions?

I know that bookkeepers worry about cents when dealing with the national budget, but the OP is not about bookkeeping.
 
  • #12
Svein said:
How is it possible to design software dealing with quantities and somehow ignore negative numbers? Since you are dealing with possibly large numbers, you routinely use FLOAT (or double in C terminology). The only case I would use unsigned would be in "counting" variables.
Have you ever written any financial software? As @rbelli1 says you NEVER use float (well you might do once, right before you are sacked).

Early systems (COBOL, PL/1, DB2) used decimal data types with fixed precision. Now we tend to use integers represeting cents/pence etc. This can either be signed or unsigned with a separate flag to indicate sign (a Debit/Credit flag or Dr/Cr).

Svein said:
But when calculating gain/loss of stock transactions, who cares about fractions of a cent when the amount is in the millions?
Everyone. In forex (foreign exchange) prices are quoted in pips, 1pip = 0.01 cents. At the moment I can see buy/sell prices on USD:GBP of 1.22039/1.22048, that's a spread of 9 pips - that's for a retail trader on a mini-lot (GBP10,000). If I were a bank that spread would be much less. Now it's 1.22131/1.22145 - the price has moved and the spread has widened to 14 pips in a matter of seconds.

Svein said:
I know that bookkeepers worry about cents when dealing with the national budget
Absolutely the opposite, the national budget is subject to so much uncertainty and assumption there is no point in using more than 3 significant figures. Models typically use either floats or integers representing base currency x 1e6 (millions).
 
  • Like
Likes rbelli1 and FactChecker
  • #13
Svein said:
who cares about fractions of a cent

With float your model can approximate the target object. If you use fixed precision then you can make your model exact to the object. You get better precision for less than free (integer types may be quicker to calculate than float types and almost certainly cost less electrical power). This is a case where in the choice better|faster|cheaper you get all three.

BoB
 
  • #14
rbelli1 said:
With float your model can approximate the target object. If you use fixed precision then you can make your model exact to the object. You get better precision for less than free (integer types may be quicker to calculate than float types and almost certainly cost less electrical power). This is a case where in the choice better|faster|cheaper you get all three.

BoB
The range and precision of a double in C is the range and precision of an IEEE754 binary64 (https://en.wikipedia.org/wiki/IEEE_754) which means that it has a precision of about 16 decimal digits and a range of about [itex]10^{-308} \dotso 10^{308} [/itex]. Now tell me how you are going to do that better using integers (which are standard at 32 bit).
 
  • #15
A bit unrelated. Today I read that when Nadia Comaneci received the first ever perfect 10s in gymnastics, the scoreboards coudn't cope and showed it as 1 instead.
 
  • Like
  • Informative
Likes Klystron and FactChecker
  • #16
Svein said:
The range and precision of a double in C is the range and precision of an IEEE754 binary64 (https://en.wikipedia.org/wiki/IEEE_754) which means that it has a precision of about 16 decimal digits and a range of about [itex]10^{-308} \dotso 10^{308} [/itex].
Not taking sides here, but a range that is greater than the 16 digits as integers is not a usable advantage if round-offs are to be avoided.
 
Last edited:
  • #17
Svein said:
Since you are dealing with possibly large numbers, you routinely use FLOAT (or double in C terminology).

I can see where you might think that, not being familiar with how computers work nor business programming, but floating point operations are even more inefficient than packed-decimal, which - for a number of reasons - continues to be the best choice for many categories of data (mostly business, but some scientific applications, too).

The only case I would use unsigned would be in "counting" variables.
Which puts you firmly in the pack with the OP issue.

For instance: the shipment of widgets that got here this morning hasn't been entered into the system yet, but we've sold a bunch of them so the computer says there's -5 widgets in stock. That sort of thing.
 
Last edited:
  • #18
hmmm27 said:
I can see where you might think that, not being familiar with how computers work nor business programming, but floating point operations are even more inefficient than packed-decimal, which - for a number of reasons - continues to be the best choice for many categories of data (mostly business, but some scientific applications, too).
Where did you get those ideas? Even if I mostly did hardware-level programming, I have done my share of business programming too. If you are interested I can provide a list.

And packed decimal was not appreciably faster than floating point when floating point co-processors were invented.
hmmm27 said:
For instance: the shipment of widgets that got here this morning hasn't been entered into the system yet, but we've sold a bunch of them so the computer says there's -5 widgets in stock. That sort of thing.
Yes - which means that you should not use unsigned integers in database fields (I do not even know if you have that option).
 
  • #19
Svein said:
Where did you get those ideas? Even if I mostly did hardware-level programming, I have done my share of business programming too. If you are interested I can provide a list.

I seriously can't imagine why you'd use any representation other than BCD, for most things involving money, or any other type of numeric data which spends a fair amount of time in the "human" domain.

Screen to disk, disk to screen, disk to printer ? straight translation, no numeric conversion involved. Want to run percentages or permillies ? just (re)define the field as such in the source code, no actual dividing by 100 or 1,000 required. Need to read the core or a disk ? The figures are right there in human-readable form.

Yes - which means that you should not use unsigned integers in database fields (I do not even know if you have that option).

Depends on the database language, I imagine. Obviously there are fields - which aren't as prone to corruption as a "stock on hand" type - where unsigned is the best fit.
 
Last edited:
  • #20
Svein said:
Where did you get those ideas? Even if I mostly did hardware-level programming, I have done my share of business programming too. If you are interested I can provide a list.
I really wouldn't admit that if I were you. There are three important reasons for this:
  1. Floats don't add cents properly. See this fiddle - and that's with 64 bit arithmetic where the effect is less marked than it used to be when all we had was single precision. Modern financial applications tend to use this rather than packed decimal; as you can see from the fiddle you can store a maximum signed value of about 60 trillion, accurate to the cent, in a 64 bit integer.
  2. Floats don't store exactly. Apart from being easier to code, I know the first query works, does the second? How will you write a test to prove it?
    SQL:
    UPDATE products SET price = price - 1 WHERE (MOD(price, 100) = 99);
    SQL:
    UPDATE products SET price = price - 0.01 WHERE (price - FLOOR(price, 0) = 0.99);
  3. Integer arithmetic is fast and can be provably implemented - the Pentium floating point bug is infamous, but have you ever heard of an integer arithmetic bug?
Svein said:
Yes - which means that you should not use unsigned integers in database fields (I do not even know if you have that option).
That may be your opinion, but again it puts you in a small minority. You cannot order a negative quantity of anything, order quantity should be unsigned, as should many other columns (an order of a negative quantity is a return and if that is a business requirement then it needs to be handled separately). You are right that stock quantity should be allowed to go negative but that does not mean that everything can.
 
  • #21
Sorry, we are talking about different things. Your examples are taken from database queries, I am talking about programming. I studied numerical analysis back in 1964, so I am quite familiar with numerical problems related to finite word length.

As to database fields, I have done some research on real-time databases (a betting system where I was supposed to update the database contents every 15 seconds and create a set of web sites every 5 minutes). That meant finding out what type of fields I could create and what response I could get from the database engine (large, well-known databases had no problem running my queries as fast as I needed, but they would choke on the update requirements).

My point is: Not every business program can be solved using a database. I have written:
  • A program (using an Intel 8008) running a noise analyzer in order to calculate the noise damping characteristics of a sound isolated wall
  • A program (using a Z80) to calculate the net heat exchange between a proposed building and the environment using real logged weather data.
My main programming field has been on the technical side, though, such as robot control, time synchronization, factory automation etc.
 
  • #22
Klystron said:
Y2K as a bug had been overblown by the press.
It was also massively overblown by fat cat consultancies who earned huge fees updating perfectly good code. After 2000 they invented another 'emergency' to deal with. I heard of one Asian country who made no preparations. They said that they would see what happened on the international date line on the great day, and quickly fix any problems. There were none.
 
  • #23
George Keeling said:
It was also massively overblown by fat cat consultancies who earned huge fees updating perfectly good code. After 2000 they invented another 'emergency' to deal with. I heard of one Asian country who made no preparations. They said that they would see what happened on the international date line on the great day, and quickly fix any problems. There were none.
I don't believe there were no problems unless they used software that other people had fixed. Although I agree that it was overblown, it definitely was a significant problem.
 
  • #24
Svein said:
My main programming field has been on the technical side, though, such as robot control, time synchronization, factory automation etc.
Such scientific / engineering software may be quite different from financial software.
 
  • Like
Likes pbuk
  • #25
Svein said:
Sorry, we are talking about different things. Your examples are taken from database queries, I am talking about programming. I studied numerical analysis back in 1964, so I am quite familiar with numerical problems related to finite word length.
Only the second example was a database query actually, but I agree that we are talking about different things.
Svein said:
A program (using a Z80) to calculate the net heat exchange between a proposed building and the environment using real logged weather data.
Interesting that you mention the Z80, it was one of the few microprocessors to implement BCD arithmetic - I wonder why they bothered with that :biggrin:
 

1. What is "Y2K By Any Other Name ...."?

"Y2K By Any Other Name ...." is a phrase used to refer to the potential technological problem of the year 2000, also known as the "Millennium Bug". It refers to the fear that computer systems and software would not recognize the change from 1999 to 2000, causing widespread malfunctions and disruptions.

2. Why was there concern about the Y2K problem?

There was concern about the Y2K problem because many computer systems and software were programmed with only two digits for the year (e.g. 99 for 1999). As a result, when the year 2000 arrived, these systems would interpret it as 1900, potentially causing errors and malfunctions in various industries and sectors.

3. How did the world prepare for the Y2K problem?

To prepare for the Y2K problem, companies and organizations invested significant time and resources to update and fix their computer systems and software. This process, known as "Y2K compliance", involved identifying and addressing potential issues and testing systems to ensure they would function properly when the year 2000 arrived.

4. Did the Y2K problem actually occur?

No, the Y2K problem did not occur in the catastrophic way that was feared. This is largely due to the efforts made to address the issue before the year 2000 arrived. While there were some minor glitches and errors reported, overall the impact was much less severe than anticipated.

5. Are there any lessons learned from the Y2K problem?

Yes, the Y2K problem highlighted the importance of regularly updating and maintaining computer systems and software to avoid potential issues in the future. It also demonstrated the potential consequences of relying heavily on technology and the need for contingency plans in case of unexpected disruptions.

Back
Top