For a few years I was contemplating ideas about the future. Especially those that give some insights about predictability of future events. When I was working in a cryptocurrency company I was exposed and became interested in markets. I became hooked on the ideas of predicting the behavior of them. I learned a great deal about the game theory, stochastic processes and sparks of insight that launched me on a journey to gain more knowledge about the mechanics of markets and generally patterns that guide time series data analysis and predictability.
If you don’t know what time series data means, you can imagine for example temperature readings outside your home that you take note of systematically with specified period like a minute, an hour, a day, a week, a month, etc. You make measurements of the specific variable like temperature, pressure, solar irradiation, etc., every minute and then given enough data you can make some predictions about seasonality, periodic trends caused by seasons and average temperatures during day or night. After all you know that during winter months the temperature is most likely to be lower than during summer, at least in the northern hemisphere. So given enough data you can determine that average temperature will be fifteen to twenty degrees higher in July than in January.
Since then I was experimenting with different techniques to see if any of them yields some better odds than just pure random chance. Three years ago I have discovered something interesting that lead to me to develop some of the beliefs I hold currently and mostly assume are true with my limited understanding. But before I explain what idea exactly I had there’s a need for explanation of some things that allowed me to devise this idea that maybe very ineffectual, but might or might not lead to something great or at last spark of some discussion and possibly lead to some new discovery in the future.
I was always interested in physics, philosophy and the nature of consciousness. It’s fascinating for me to read and learn about new discoveries in physics, especially in the quantum mechanics realm and recent discoveries that hint that quantum effects might have a large effect on how our brains and consciousness works. In neuroscience there are some hints that our brains use quantum effects like tunneling and perhaps even quantum entanglement of particles that allow our brains and minds to function at all.
I have to admit that I am no expert in those topics. I am simply fascinated by them and I try to integrate this knowledge to my everyday life and beliefs. I like them grounded in solid science and always have a skeptical point of view, reinforced by different perspectives, opinions and new empirical evidence. However despite knowing that there will be new inputs that might change my view of some things, at any given time I can only devise hypothesis and try to verify them with the knowledge and evidence that I have. But I’m always willing to explore ideas that seem fringe or far out. I take a stance that if you limit yourself to what you already know and not try to poke those cracks in reality, you are essentially regressing, as the most motivating factor in ones life is novelty. And I always seek it and challenge my established views and opinions, to break down and shake my stable worldview. Honestly, despite experiencing much resistance from external and internal factors, it never turned out to be bad. It always lead me to new insights and ideas.
And this is fun. Exploring new things. Shaking off everything you thought to be true, to broaden your perceptions and question your reality. After all, even if truth is not pleasant, it is possibly the truth (not saying objective as I am not capable to determine if such thing even exists) so one has to accept it or live in a blissful ignorance.
So let’s talk for a while about mundane topic of signal processing and the Fourier Transform. I understood what Fourier Transform was, and that it allowed to decompose the raw signal into time frequency domain to show a spectrum of frequencies of the signal. When I first learned visually and intuitively that Fourier Transform is effectively the first mathematical machine that was invented I was absolutely astounded and honestly, blown away. Every signal could be decomposed into constituting frequencies of sine waves and show you exactly how much of each frequency component contributes to the overall signal. This however comes with one problem. Fourier Transform gives you accurate information about frequencies in the signal spectrum, but you have no way of pointing where exactly these frequencies occur within the signal.
Okay, so why is, you might ask. It comes down to some fundamental principles and physical laws that I see reoccurring time and time again in different physical and informational phenomena, that essentially boils down to Heisenberg Uncertainty Principle. Setting aside a little philosophical thought that the only thing in life and universe that is certain is uncertainty, we can say that when you try to measure a signal you have two variables: time of occurrence and frequency of constituting waves, sinuses or sub-signals. You can either have absolute certainty on one, but then not the other and vice versa.
There is of course a little hack that allows you to analyze the signal with increasing precision in both of these analysis domains. You can use this ingenious mathematical trick called wavelets to process signals, with different time and frequency resolution to produce not one dimensional frequency map, but a two dimensional heat map, that correlates time and frequency of the signal, so it’s easier to pinpoint exact frequency appearance within time domain.
However, this gives you only moderate clues about when, where and how the signal looks like. And most of all you are working on data that is already set in stone (already happened). This might give you some insights into how the signal will develop in the future and many times, if your model is working well, it might give you good hints about some predictions with regards to the signal developing in the future. But since most of the time series data is essentially a stochastic, random process, there’s a high degree of uncertainty for your predictions. That might be alleviated to some extent by seasonality and trend analysis, but essentially underneath you have to assume that time series data like the market is essentially a random walk and you can’t predict it with 100% certainty. You can only make some statistical assumptions which are better or worse.
So, what can you do about it? Well, I found a probable way out of this. So let’s think of the signal in the given time period. What can we say about it? We can probably try to average it to it’s constituent sinuses or base waves in the frequency domain. This is called the polynomial fitting. If you can assign a highly correlated wave (compound sinuses) to match and fit the signal then you can probably to high degree predict with ease the development of the polynomial function in the future. Of course this prediction will degrade with time, but for short-term predictions it should be fairly accurate.
Let me digress here a little bit, before I expand on this topic further. When I first thought about all these things, I encountered a concept of retro-causality. It appears in physics that many of the theories and physical tools we have at our disposal, are time-reversible. It’s a fact. Quantum mechanics and particle interactions could be simply visualized by Feynman Diagrams. And guess what? Within them we it’s normal to see particles interacting with other particles that supposedly go back in time. We have also already proofs that particle observation (measurement) results in establishing determination of a state of particle in the past with experiment of Delayed Quantum Eraser with entangled particles.
This is not yet proven or an established interpretation or even a theory in physics, but if you take a look at the philosophical concepts of eastern scholars, like that the only thing that matters is perpetual now, you can try to imagine what it would be like if not only past influenced your current state of being, but also the future events influence the past? I won’t go into much detail here as this is a broad topic to explore and it’s not my intention to do here, but consider this. What if your future self is influencing where you are going? What if there was a way to communicate information from now to your past? Or from your future to your present? Is it even possible?
Well, many physical processes work the same in both directions of time and the results hold consistent. Of course there’s entropy and as far as we know and anything that could travel backwards in time would be faster than light in essence, which we now think is nonphysical. But. There are hints that antimatter might be traveling backwards in time. And some other physical phenomena like quantum entanglement essentially breaks the cosmic speed limit of light speed as it’s a spooky action at a distance that happens instantly, not obeying the speed of light. Yes, we are fairly certain that these cannot convey any meaningful information, but it keeps you wondering, if there could be a way to determine the future from the signals originating from the future.
Given the strange nature of quantum interactions and deduction from Feynman Diagrams of particle interactions, might give you counter-intuitive results that indeed information could be carried over from the future to the present, just perhaps with a phase shift of the wave propagation from the point of origin of the signal to the past or your present.
Most physicist however think that retro-causality is non physical, but I wouldn’t dismiss it already as there are some clues that it might be realistic and true. We just don’t know yet.
So, given all this what I have said, is it possible to even predict the future of the essentially random stochastic process? Perhaps it’s not very elegant solution, but I think there is a way. At least to very high degree of probability. As the nature of the Universe, especially the quantum microscopic world is essentially random, how you could predict randomness? Seems impossible.
But on the small time scales, you might reach a fairly good degree of certainty by using the oldest and the most simple prediction algorithm. Brute force.
Let’s assume we have a signal consisting of variable that changes over time. We can use tools given to us by mathematics and physics to try to model the signal to make a high degree polynomial fit, utilizing also the Wavelet transform or Fourier transform to support our polynomial fitting of the signal to the function. This would allow us to model the probabilistic future outcomes of the polynomial fit for the signal at short time-scales. Of course this fitting would be very inefficient, but in theory if we define a vast array of functions to compare our polynomial fit of the signal, we should find a wave or a compound sines forming the signal that allows us to predict future signal behavior based on a rather simple equation. Of course this is inefficient, slow and as all brute force algorithms, computationally intensive, but in theory it should work on short time scales.
So, maybe this does not constitute a true retro-causal interaction, but in my opinion it is as close as we can get to predicting the signal that is essentially random at nature.
For all this to work there’s a concept of coherence of two waves. In physics and especially quantum mechanics coherence allows you to compare two waves in terms of their similarity to each other. Of course for this to work this algorithm would need a large set of functions, including those chaotic ones (exponent powers, etc.) and a lot of trial and error on basic and not so basic transformations, like scaling, translating and perhaps even rotating. And it has to work a vast search space for a finite time series period. It would be very inefficient as an algorithm, but it’s a start.
Intriguing as it is coherence and Fourier Transform gives some interesting clues about functions of the data that are non-stationary. Did you know that when you feed these with a chaotic function that is non-stationary, the overall slope of the frequency domain would reflect the time-dependent change of the function values? So there might lie a clue for predicting the coherence of a polynomial fit of the signal to the function of interest.
If you think about it, there’s nothing that can prevent such an algorithm to work and I have done tests with the code I have developed, to show that finding a coherence of polynomial fit of the signal to the baseline function is indeed possible. The hard part is of course finding a high-degree polynomial fit for the signal, but you could probably do away with a good approximation.
So, there you have it. It should be possible to predict future development of the signal, by trying to find coherence between it’s polynomial fit and a vast search space of transformed base functions. It would be inefficient, but it could find solutions to a good degree and with a lot of computing power it should be more than likely to beat the odds of random chance.
And one more food for thought. If you think about this mechanism, if it’s possible to take a peek at the future with a high degree of certainty, it gives you some philosophical entertainment about the nature of consciousness and reason. What if our consciousness is not only a product of the past, but also of the future? What if our consciousness can only really function with retro-causal information? What if the future influences ourselves as much as our past influences us? If consciousness incorporates this mechanism one way or another, even if we do it unconsciously, what would it mean for causality? Or perhaps that given the strange nature of some interpretations of quantum mechanics or the reality itself, it is essentially true, what would it mean for you to exist?
I’ll leave that up to you.
