Forex trading from engineer with good background in maths-NN

[Konstantinos]

Hi all.

I am engineer with a good mathematical and neural networks background:

In Mathematics I know very well all the aspects of Mathematical Analysis (including Lebesgue Measure and Integration), Numerical Analysis (including numerical methods of Differential Equations), Linear and non Linear Algebra (with applications in Signals and Systems, Discrete Time Signal Processing, Control Systems Design, Computer Vision) and Mathematical Logic. Moreover I know the most popular optimization algorithms.

In Neural Networks and Artificial Intelligence, I know very well all their aspects, for example Dimensionality Reduction, Multivariate Visualization, Classification Algorithms, Shallow Networks (Classification, Regression, and Clustering), Generalized Models, Mixed-Effects Models, Analysis of Variance, Fitting Distributions, Deep Learning Training, Pretrained Models, Accelerated Training with GPUs, and Preprocessing and Postprocessing.

So my question is the following: I have decided to get into the forex online trading. How to do so? What are my first steps? What are my first expectations? For example if I start tomorrow, what I have to expect the first month, the first 6 months and the first year?

All the google is full of advice for people with Middle School Mathematical level and it kind of confuses me. The first advice is to focus on indicators (like RSI, MACD etc), to create automated trading robots (Expert Advisors etc) etc. And it is expected to have profits. Also metatrader 5 has a manual neural network choice, in which you can optimize your strategy (training) and then you can validate it (forward)

About me, I have to focus on this advice, or is there a different way to apply my knowledge in that field? For example can I take the historical data of eurusd and apply to it all the signal processing and neural networks toolboxes of Matlab and Python?

Simply said, what steps I have to follow on the beginning and what my expectations should be?

Thanks in advance.

 

[BWV]

Question - what do you think any of that will result in you making excess returns when there are already hedge funds and prop desks using (or have tried) all these algorithms and have far greater experience and expertise than you?

 

[Konstantinos]

Hi BWV. I believe that my i7 core with 8 threads and 16 ram is enough for an annual return 1.1 - 2.0. Also, we can rent supercomputer services from Amazon or Google, to train a neural network for all the ticks over the past 10 years, to make it able to predict the future (at least for a small period). Even 1 week prediction would be enough...

Moreover, Matlab has many tools:

Financial Toolbox: https://www.mathworks.com/products/finance.html
Financial Instruments: https://www.mathworks.com/products/financial-instr...
Risk Management: https://www.mathworks.com/products/risk-management...
Trading: https://www.mathworks.com/products/trading.html

All of these are not just "air without meaning".

I am looking for guidelines for my first steps, for my starting point. For example, someone would say, that in my first steps I have to focus on eurusd with H1 or M5 time frames, and I should try to train a neural network in the last 3 years, using this training method, using real ticks and I should expect a mean monthly return of 1.015 implying an annual return of 1.2. Or that I should use the available indicators: https://www.mql5.com/en/code/mt5/indicators and I should create my own a strategy. After that I should optimize it the last 3 years (2 years training and the last year validation - forward), manually by using the available optimization techniques from brokers or trading platforms. Or in my first steps, I just have to focus my strategy to be a kind of coin toss, not losing money (50% profit trades - 50% loss trades) and the 1.2 annual return is more advanced?

Please, I hope there is someone who can guide me.

 

[berkeman]

Thread closed for a spam setup investigation...

 

[berkeman]

Thread is re-opened after a PM conversation with the OP. Thread is being monitored. Could turn into an interesting thread, actually...

 

[BWV]

Hi BWV. I believe that my i7 core with 8 threads and 16 ram is enough for an annual return 1.1 - 2.0. Also, we can rent supercomputer services from Amazon or Google, to train a neural network for all the ticks over the past 10 years, to make it able to predict the future (at least for a small period). Even 1 week prediction would be enough...

Moreover, Matlab has many tools:

Financial Toolbox: https://www.mathworks.com/products/finance.html
Financial Instruments: https://www.mathworks.com/products/financial-instr...
Risk Management: https://www.mathworks.com/products/risk-management...
Trading: https://www.mathworks.com/products/trading.html

All of these are not just "air without meaning".

I am looking for guidelines for my first steps, for my starting point. For example, someone would say, that in my first steps I have to focus on eurusd with H1 or M5 time frames, and I should try to train a neural network in the last 3 years, using this training method, using real ticks and I should expect a mean monthly return of 1.015 implying an annual return of 1.2. Or that I should use the available indicators: https://www.mql5.com/en/code/mt5/indicators and I should create my own a strategy. After that I should optimize it the last 3 years (2 years training and the last year validation - forward), manually by using the available optimization techniques from brokers or trading platforms. Or in my first steps, I just have to focus my strategy to be a kind of coin toss, not losing money (50% profit trades - 50% loss trades) and the 1.2 annual return is more advanced?

Please, I hope there is someone who can guide me.

I will guide you - try something else, this is a loser's game

Earning a return above the small carry (the short term interest rate you can earn holding a currency) is a zero-sum game, the net gains of all forex traders above short term interest rates is zero

Forex trading for ordinary schmucks like us is a notorious area for scams

You are trading against professionals with lower transaction costs, better computers, more experience, and better math credentials

What makes you think you can win this game? (and in reality the professionals do not make great returns, certainly nothing like 120% per year).

Also what makes you think that future exchange rates are predictable from past prices? (they are slightly - price momentum has a weak effect, but it is unreliable and delivers nothing like the returns you are describing)

 

[StoneTemplePython]

Or in my first steps, I just have to focus my strategy to be a kind of coin toss, not losing money (50% profit trades - 50% loss trades) and the 1.2 annual return is more advanced?

How well do you understand (negative) convexity? I don't think FX has much do with 50:50 trades breaking even. Magnitudes matter, a lot. There's a lot of leverage and well known peso problems in the data. I'd anticipate 'great' returns until a hiccup where you lose much or everything.

- - - - -

What makes you think you can win this game? (and in reality the professionals do not make great returns, certainly nothing like 120% per year).

given that $1.015^{12} \approx 1.2$ I read this as 20% per year -- i.e. quoting $(1+r)$ not $r$.

i.e. as in the below quote estimating a monthly compounding rate of $0.015$

I should expect a mean monthly return of 1.015 implying an annual return of 1.2.

 

[Konstantinos]

BWV thanks for your response. About the professionals you said, that's the point. I don't want to win them and i don't want to play in a rebellious or antisystemic mode. I want to get into that class to cooperate with them and to build all my life's career in this field with them. So if they have annual return 110%, managing 1 billion dollars, what i have to do to be able to prove them my abilities? 120% return, managing 50.000 or 100.000 dollars is not possible? Also they don't focus in forex so much, they have better choices.

Personally i feel that the academic knowledge of neural networks and artificial intelligence, with the last optimization methods + the use of fast supercomputers are a good combination for having the first profit in forex. And you don't want to predict all the future. Maybe just 0.5% will be enough. Also in forex, small changes in currency are multiplied by the leverage which it can 500 in Europe.

PS: So far i have something like 1.2 return in the forward validation and testing mode of the neural networks, just by using very simple strategies. And all this training is done in all the real ticks. I mean i feel that if i find some guidelines (not in forex, just in neural networks prediction) it would be better and profitable.

 

[fresh_42]

Magnitudes matter, a lot.

Exactly. There is so much money constantly running around the globe, that the only chance is to permanently guess what the others will do, and this doesn't necessarily depend on certain data, but might just be a decision to swap positions in a portfolio. Considering the trading costs, it's probably a lose-win situation, and you will not be the winner. Chances with sports bets are better. In my opinion, even Vegas and Monte Carlo would be better. E.g. the €-$ prize is basically always between 1.10$/€ and 1.30$/€. So buying low and wait will likely be better than to compete with all those really big, big, big players. One single trade of the Chinese central bank could ruin you in a second. Usual business stocks don't necessarily reflect balance data, and currencies even less. If Moody's will downgrade a certain nation just one grade, you very likely won't know in time, whereas all others will, and so on, and so on.

 

[Rive]

What are my first steps?

Simply put, you have to realize three facts.

First: you are not the first, and all your opponents has higher degree and started this game far sooner than you - some of them are even making the game
Second: forex (for the small players, I mean) is often regulated as betting, not investing category. And this is just right.
Third: the only really stable money from forex, stocks and other markets comes when you don't trade the market, but sell the market, like selling trading platforms, ideas, programs, predictions, sure methods and such. Google up the interet - the 'getting rich' business is just second to the 'health' products

... what my expectations should be?

You should expect losing all your initial capital, but getting the well-known mindset afterwards that you were just a nick away from success.

 

[BWV]

Where do forex profits come from? Why should anyone be able to consistently earn excess returns betting on exchange rates? If you make money by, say, the Euro going up vs the dollar then it is only because someone else lost on the trade. So you can’t cooperate with other traders anymore than you could cooperate with other players at a poker table. There are two major groups of institutional traders in forex that earn returns (or at least attempt to) above the carry:
Macro hedge funds (most of these are systematic momentum strategies, which is likely what a NN is going to replicate if you feed past price data into it)
example: https://www.ahl.com/technology

Market Makers (use algorithms to replace what floor traders used to do - makes small bid / ask spreads and provides liquidity to the market - this is also what high frequency stock traders do)
example: https://qz.com/1210374/how-do-you-design-an-office-for-robots/

 

[f95toli]

Hi BWV. I believe that my i7 core with 8 threads and 16 ram is enough for an annual return 1.1 - 2.0.

You would be competing with people who run ASICs and large FPGA systems. There is no way you or anyone else can compete when it comes to resources unless you have a large financial institution backing you.
Just to give you an idea of what you are up against: some companies here in London have started setting up offices to the west of the city (near Slough). The reason is that this is were some of the fibre optic cables that go underneath the Atlantic emerges. Since the speed of light is finite they can therefore shave a couple of milliseconds or so off the time it takes their high-speed trading algorithms to react to what is happening in the US (algorithms that in many cases are running on large FPGA systems situated in the basement of the building to be as close as possible to the point where the cable enters the building).

I know you are not interested in high-speed trading, but the point is that there is an enormous amount of money available in the financial section which translates into a lot of equipment and the ability to pay smart people a LOT of money (I know some traders) if they are any good.

 

[StoneTemplePython]

A quote from Howard Marks comes to mind here:

The more challenging and potentially lucrative the waters you fish in, the more likely they are to have attracted skilled fishermen.

The point is somewhat obvious in a way if you think about profit motives and put a Bayesian hat on.

OP seems to be suggesting he can find a niche that is too small for really big players to find profitable and is amenable to deep learning, (and other players by and large aren't using deep learning already in this space).

If true, it's a fair point. But other than maybe some very exotic highly illiquid (collapsed?) currencies, I don't think there's anything so small that big skilled fishermen are being deterred in FX. And for such illiquid things there won't be much by way of data. And FX tends to be inherently leveraged -- it's also extra difficult managing leverage against something highly illiquid.

Exactly. There is so much money constantly running around the globe, that the only chance is to permanently guess what the others will do, and this doesn't necessarily depend on certain data

OP's idea is unfortunately a semi-common one in the Bay Area -- people who know a bit about deep learning and very little about finance seem to have it. What I try to tell them is consider typical Machine Learning applications, e.g. image or speech recognition. In a general (i.e non-adversarial setting) you have a 'reality' that is independent of your inference, and you're just trying to get the machine to get better at making predictions about said reality.

In finance, your 'reality' is a function of other players using inference and other tools -- i.e. the reality isn't particularly stable and is hardly independent of the cutting edge analytical tools that you may think you have on hand. It's a very different game. I then tell people to google Renaissance Technologies.

 

[borson]

Very interesting.
I do not understand why people are so pessimistic here about Trading.
It is very feasible to make a 10-20% monthly return from it. (Depending on what you trade with). Futures are the most professional, regulated, "cheap" (in terms of comissions) and profitable products to trade. Those who live off of this trade with futures. I know some people personally who do so. In fact I am about to finish a course on it. One remarkable thing to mention is that it is not necessary to have 5 degrees, 70 years of experience and a quantic computer in order to make money in this. People I know who are profiting from it do not even have a career related wih economics or math. It is not about having all the knowledge, but instead, about having a right/useful piece of the whole cake of knowledge, and just using it. It suffices with having a good strategy, and then applying it consistenly and with discipline.
With this being said, I would recommend you to start there, in the futures market.
About forex, well, forex is a descentralized market, this implies that you don"t have valuable data such as volume, Deltas, COTs... (It does not necessarily mean though, that you cannot use a different strategy that does not use all these data)
Well so, my first advice would be to start with futures, but nevermind if you really prefer to trade CFDs (Forex), you can be profitable anyway with the right strategy (Although there are fewer "right strategies" for forex)

2- I would not use Indicators to trade. This is from my point of view a fallacy. They are just a mathematical function of the price, thus, they do not provide any new and real data (that is, any variable the price's movements depend on)about the quotation (such as for instance, volume). I would refuse to use any strategy based purely on indicators. Bollinger Bands and MAs have a pass though. I would not use them to make trading decisions, but they can be useful to complement your view of the market.
3- It is not about playing against people, nor about playing against professionals, indeed, it"s just the other way around. Why would you go against an institution who is putting ×100000 times more money than you in a determined side of the market? Why would you sell if they buy? That millionare stake moves the market, and pulls/drags many other people with it, who, prays of the ambition of seizing this movement enter in the same direction. Then, if due to this price is moving up, why would you sell expecting to win the pulse against a millionare instituion?. (In reality this is a simplification of how markets work and of how institutions make their orders, in reality this is a little bit more complex, but just a little bit, once understood, one can see where rich hands put their money on, and thus put theirs on the same side too)

To sum up: The best place to trade in is the futures market. You can make money in Forex/CFDs too, but it has worse conditions.
If it's just about making an EA, I would recommend you to do it first for the Forex/CFDs market.

4-I am fascinated with your project. I would really like to know how it goes and to help you with whatever you need. In fact, I am currently working on an EA too. If you want to ask me any thing, just e-mail me; <<Moderator's note: e-mail address removed. Please use the forum's "Converstations" (private message) feature instead>>
In fact I'd like to ask you some questions related with math too

 

[Rive]

I do not understand why people are so pessimistic here about Trading.

Because of experience and resposibility.

I too know some people who are doing the trading and doing courses as secondary (usually it is the other way - guess why). Some of them has even more hilarious monthly return than what you mentioned. But the starting point of their courses is, that the mindset required for successful trading can be developed for one in a ten thousand - the rest just can't become (good) traders. Also that mindset needs continuous maintenance, and actually the worst enemy of that mindset is success.

So, if some of us sayin' here that yes, it can be done, then it'll just not work for ninety-nine hundred ninety nine in ten thousand: those will lose. If they are lucky then they will lose right after they start.

If we say here that it can't be done then we actually do lie - but it'll perfectly fit for the vast majority.

Of course there is always somebody who will bring up some high returns.

 

[Konstantinos]

@borson, thanks for the encouragement, I sent you an email...

I have good news. I talk with some hedge funds for some positions and it is going well. With one of them, we are in the final steps for cooperation. Finally, being 30 years old, I have found out what I want to do in my life.

The most important sense, in the point that I am now, is Kalman. For example Kalman training for neural networks or Kalman filtering: https://en.wikipedia.org/wiki/Kalman_filter

PM if someone wants to keep in touch... You are all welcome...

 

[fresh_42]

I do not understand why people are so pessimistic here about Trading.

I cannot answer for people. My answer is, because I know how trading is done professionally.
You won't compete with Bolt on a 100 m track, won't you?

 

[bhobba]

Lets take what many think is the worlds greatest trader, the very good mathematician, James Simons, who runs Renaissance Technologies. As part of that he overseers the Medallion Fund - likely the world's greatest money-making machine. Medallion is open only to Renaissance Technologies roughly 300 employees, about 90 of whom are Ph.Ds, as well as a select few with connections to the firm. It gets a return of I think 80% - whereas 10% pm compounded over a year would be 314% py. In Australia we have something called the pup test. Do you think the claimed returns pass that? I could answer using colorful language but its so obvious nothing else needs to be said.

I have traded and know people that do it. If market conditions are good the best you can do after tax is about 15%-20% py - and that is only if market conditions suit. Quite a bit of the time you either are out of the market or loose money waiting for it to turn. Its illegal here in Aus for a person who is not a certified financial adviser to give advice so I will not. I can however point you to well known strategies purely for educational purposes that are in no way to be taken as advice - dollar cost averaging:
https://en.wikipedia.org/wiki/Dollar_cost_averaging

And the volatility of a geared fund:
https://www.perpetual.com.au/~/media/perpetual/fund-profiles/130_pfp.ashx

Combine the two and you will see the type of returns you can realistically expect to get over the long haul. DCA allows you to set aside a certain amount from your pay and reduce overall volatility. As you can see you get superior returns in a geared fund but at great volatility. But their are pro's and cons as you can read.

But you run into the issue called by some, the failure of portfolio theory - the market may be down just when you want the money. However its about the best the average person can do. I know - I have used other methods and their after tax performance is not much if at all better - but hope springs eternal. Do not take this to mean superstars like Simons can't get spectacular performance. The market can be beaten, but spectacular returns are only available to the superstars like Simons.

Thanks
Bill

 

[BWV]

Good post by would clarify the issue of the market being down when you need money is not a failure of finance theory - the theory states that this risk - a high correlation of asset portfolio prices to changes in aggregate consumption is the whole reason for the equity risk premium. The problem with forex is that, unlike stocks, there is no real risk premium. There is a carry tied to the differentials in short term interest rates between currencies, but that is generally a small number between developed market currencies.

Most trading operations make money by either by market making - providing liquidity to large hedgers, but this is a business of scale with tiny incremental margins, or by trying to exploit momentum effects - which small traders can do (this is basically all technical analysis or machine learning on past price patterns), but this is volatile and subject to periods where it can lose substantial amounts of money

 

[bhobba]

Good post by would clarify the issue of the market being down when you need money is not a failure of finance theory

Its called by some that ie the failure of portfolio theory - its not really a failure - simply a characteristic. What you would do in practice is as you approach when you want to use it - ie retirement - you proportionately invest more in cash as you get closer. Or you can invest say 50% cash from the start. It can be handled - you just need to be aware of it and plan how to handle it. That's why in Australia you are not allowed to give advice without a license - you need to take into account the investors objectives, appetite for risk etc before investing. But I have to say being STEM guys you can probably figure it out for yourself what they will say - here is a textbook they often use (yes at one time I thought of being an adviser but was disillusioned by the whole business of advising others - you feel more attached to what you figure out for yourself):
https://www.amazon.com/dp/B005XM6NRY/?tag=pfamazon01-20

Understandably its meant to reduce risk - but not necessarily maximize return. It says you can't beat the market - but there are techniques that prove you can - like the one above that uses gearing and volatility - but you need guts to stick it out. That type of guts only come from investigating it yourself and understanding it.

Thanks
Bill

 

[bhobba]

Most trading operations make money by either by market making - providing liquidity to large hedgers, but this is a business of scale with tiny incremental margins, or by trying to exploit momentum effects - which small traders can do (this is basically all technical analysis or machine learning on past price patterns), but this is volatile and subject to periods where it can lose substantial amounts of money

That's correct. I have tried some systems like that. The issue is after tax returns compared to long term trading (not investing which I take to mean buy and hold) - with trading you do sell - but often keep shares for years. Here in Aus long term trading ie keeping shares over a year you pay tax on only 50% of profits. This means your short term trading system has to a fair bit better to begin with.

Its out of print now but the best book I ever came across on it was by an author long dead, Austin Donnelly:
https://www.fishpond.com.au/Books/Sensible-Share-Investing-Austin-Donnelly/9781875857159

He was president of the Australian investors club and his particular passion was advising people how to invest/trade without being scammed. No mega returns available, but good after tax returns that were hard to beat by more sophisticated methods and with much less risk. He also set up a financial planning firm that publishes a newsletter detailing his methods. I will not link to it - you can google it yourself if interested.

Thanks
Bill

 

[StoneTemplePython]

.. the whole reason for the equity risk premium. The problem with forex is that, unlike stocks, there is no real risk premium. There is a carry tied to the differentials in short term interest rates between currencies, but that is generally a small number between developed market currencies.

I think you're right on carry and momentum. Depending on specifics of the instruments there may be premia for Tail risks (e.g. with pegged currencies)-- smiles, smirks, and skew preferences are a real phenomenon.

Its out of print now but the best book I ever came across on it was by an author long dead, Austin Donnelly:
https://www.fishpond.com.au/Books/Sensible-Share-Investing-Austin-Donnelly/9781875857159

This reminds me, there is a nice book by Antii Ilmanen called Expected Returns that picks through sources of returns. It's a bit technical but does a nice job decomposing different investment tactics/strategies and I like decompositions, so I suppose that gives a clue as to why I like the book.

I would hope OP would read this book first before embarking on FX trading, though I've noticed a bizarrely common phenomenon where people want to stake a major portion of their wealth on some financial thing and yet are deeply resistant to putting in even 10-20 hours of (difficult) background research first. There's an old joke that betting on horse racing never really took off in Vegas because it took too long for people to lose their money. Maybe it's a related point.

 

[Vanadium 50]

I've noticed a bizarrely common phenomenon where people want to stake a major portion of their wealth on some financial thing and yet are deeply resistant to putting in even 10-20 hours of (difficult) background research first.

And have crazy expectations on returns. "If I just double my money every minute, I can retire as a millionaire tomorrow!"

 

[momolo]

The readily apparent:

1) Learn the different forms of money management & you will discover the only one that makes sense which will get rid of interest in the others.
2) Analyze your chosen currency moves that allow you a profit that you can stomach on a regular basis for your bankroll.
3) Trading small timeframes with most brokers will put you in the red with high frequency trading because of lack of volatility & you will have firms trading in front of other customers known order flow.

If you want to be at the front of the pack...you are going to have to go to work as an intern at a large firm & learn what they are doing & look for improvements that
occur to you.

It should be obvious to you there are no sane Phd's that are in this business laying out their systems on the internet when it comes to high frequency forex.

If you are satisfied with making eventual very good money & are patient building your bankroll...you can train yourself to see lower frequency high percentage long & short trades in forex (Riding on the backs of whales). Most people don't have the patience for it, it doesn't rely on the computer doing the analysis & most importantly they don't know how to get out of a trade.

I would try to intern at your dream fund & make the most of it. They will have money to eventually make it worth your while if you are good.
But you need to get in their ecosystem. Why rebuild the wheel when you can meet the inventors & possibly improve it?

 

[CWatters]

I have some decades experience investing in a variety of asset classes. If there was one thing I would like to eliminate it's the effect of what I call "events". You can do extensive research before investing and watch your purchases grow steadily over months or years...only for all the good work to be undone by a politician with a Twitter account.

 

[bhobba]

I have some decades experience investing in a variety of asset classes. If there was one thing I would like to eliminate it's the effect of what I call "events". You can do extensive research before investing and watch your purchases grow steadily over months or years...only for all the good work to be undone by a politician with a Twitter account.

There is a phenomenon called regression to the mean. That's the time to buy - but you need faith in that well known statistical principle. You need guts - which many don't have.

There is another method that tests have shown still works:
https://www.amazon.com/dp/0470624159/?tag=pfamazon01-20

You need to actively manage it. But you know what the author found - people don't really want to spend time managing it - they simply want to set and forget. Human psychology has a big impact on your returns. That's why the one I mentioned with DCA and a geared fund is easy to practice - its set and forget - but you still need guts - when it makes large falls you have to grit your teeth and still regularly put your money in.

Of course if you want to trade for a living then you will likely expect to be more active. In fact you will likely have a number of systems going at once. I did when I traded - but unless financial markets really float your boat you get sick of it. I had retired and thought - what am I doing this for - really now is the time to spend what you have earned over the years.

Thanks
Bill

 

[CWatters]

You can only buy if you have cash or other assets you are prepared to sell. I always find it harder to decide when to sell.