Wall Street's Research Jobs Are the Most Likely To Be Upended By AI (qz.com) 66
An anonymous reader quotes a report from Quartz: Research analysts are the most likely employees on Wall Street to find themselves working with -- or being replaced by -- robots, according to a survey by Greenwich Associates. By next year, some 75% of banks and financial firms will either explore or implement artificial intelligence technologies, harnessing a variety of digital services to extract insights from mountains of data. While AI is probably near the peak of its hype cycle, several factors have helped it gain traction in recent years, according to Greenwich. Billions of images and documents are now available online for training computers to spot patterns and other high-level tasks. Advances in graphical processing units, which are adept at the kind of data crunching required by AI, are making sifting through daunting datasets much easier. The cloud has also made it cheaper for researchers and startups to boost their computing power to service sophisticated AI-enabled systems. AI makes sense for financial research, as machines can crunch reams of data more quickly than human analysts and, with the right data, identify obscure correlations and patterns.
Good (Score:1)
Good
Re: (Score:1)
Re: Good (Score:2)
It may not be good for a subset of the bank's employees, but those guys have made it into "the club" already. Their big fear is that their new job will *only* pay them double what they're worth instead of triple.
Most relevant is, is it good for society? That is tougher to answer without seeing exactly how AI gets implemented. The devil is in the details, as they say. But if they come to a juncture where the AI could either be tuned for maximal fairness, or maximal
Re: (Score:2)
"Most relevant is, is it good for society?"
No, this is for complex financial instruments that benefit people who invest billions, this isn't for the proles who are in debt.
Eh, maybe (Score:5, Interesting)
Problem with using AI in these scenarios is that it's really good at finding correlations in what you tell it to look at. So maybe it finds correlations between interconnected stock prices, or maybe futures and trading volumes, or the consumer price index and stock prices of certain retail stocks, things like that.
When everyone has AI's doing this, the margins get eaten up pretty quick, since everyone is getting the same results and takes the same positions.
The areas you make money on are finding the niche correlations. A nationalistic dictator takes over some African country and shuts down rare mineral exports causing a spike in prices. A geothermal plant in Iceland goes down, shutting down it's aluminum smelters and aluminum prices rise. Those are the things AI sucks at.
Re: (Score:2)
They don't have to win constantly to turn a profit. Those things you mentioned happened rarely, and all the bots need is a 60 to 70% win rate. Winning more than you lose is the only way forward.
Re: Eh, maybe (Score:5, Interesting)
There are some one-in-a-lifetime trades where people pour in really high stakes, like the one made by Soros in the GBP crash last century. On the other hand, stuff like brexit and Trump's win were really good bets even if you predicted their odds to be 50:50 - because disproportionately few people bet money on the alternative outcome.
I agree with GP - a lot of these roles would benefit from the increased productivity aided by good statistical tools, where one equity researcher has to work with or become a so-called data scientist to produce better insights.
Re:Eh, maybe (Score:4, Insightful)
They don't have to win constantly to turn a profit. Those things you mentioned happened rarely, and all the bots need is a 60 to 70% win rate. Winning more than you lose is the only way forward.
Yeah, and when that level of winning pans out and they get greedier, they pay for the development of AI that understands a little more about the real world. And when THAT pans out and they get even greedier, they'll okay the development of AI that actively interferes in the real world to produce situations that profit can be made from.
Then it'll be too late.
Re: Eh, maybe (Score:2)
Re:Eh, maybe (Score:4, Informative)
And they play off the general rise in the stock market as their ability to make people money.
That is one way, but another trick is to launch dozens of funds, and shutdown those that lose money. So if you start 32 funds, and purely by chance half beat the market after a year, so you shutdown the other 16. After two year, you have eight left, after three years, you have four, ... and finally after 5 years, you have a fund that beat the market five years in a row, which you can then promote as obvious proof that you are smart at picking stocks.
Re: (Score:3)
Those are the things AI sucks at
...for now. Those are the AI goals the big banks keep in mind, and are working on heavily.
Fundamental (Score:3)
They can work on it all they like, it's a fundamental problem with AI that people have been working on for *decades* and haven't got much closer to.
That problem is context. AI is terrible at it. The problem is it only understands what you program it to understand. So it lacks "common sense" knowledge skills.
The classic example is Abraham Lincoln giving the Gettysburg Address. You can feed that information into an expert system - that Lincoln gave the Gettysburg address at such and such a time / place / etc.
Re: Eh, maybe (Score:2)
When everyone has AI's doing this, the margins get eaten up pretty quick, since everyone is getting the same results and takes the same positions.
And that's a problem? I thought this was how perfectly functioning markets are supposed to work.
Re: (Score:2)
Problem with using AI in these scenarios is that it's really good at finding correlations in what you tell it to look at. So maybe it finds correlations between interconnected stock prices, or maybe futures and trading volumes, or the consumer price index and stock prices of certain retail stocks, things like that.
Remember the episode of Star Trek Voyager where Seven of Nine identifies all these correlations and thinks there is a conspiracy theory going on board the ship? Yea...
Re: (Score:2)
Problem with using AI in these scenarios is that it's really good at finding correlations in what you tell it to look at. So maybe it finds correlations between interconnected stock prices, or maybe futures and trading volumes, or the consumer price index and stock prices of certain retail stocks, things like that.
When everyone has AI's doing this, the margins get eaten up pretty quick, since everyone is getting the same results and takes the same positions.
The areas you make money on are finding the niche correlations. A nationalistic dictator takes over some African country and shuts down rare mineral exports causing a spike in prices. A geothermal plant in Iceland goes down, shutting down it's aluminum smelters and aluminum prices rise. Those are the things AI sucks at.
Which is why I think AI will be good at identifying where to look, but needs a human to decide what it is really revealing and what else may be useful.
Re: Eh, maybe (Score:2)
What is new about AI is precisely that it lets computers begin to usurp the decision-making role. If the only thing the AI did was collate information and present it to a human, nobody would be worried about it.
Re: (Score:2)
It's the opposite. *You* tell the computer what to look at (by feeding it data) and then it calculates an answer (=makes a decision) based on what you feed it. Just like any computer ever. AI is not magic. What is new about AI is precisely that it lets computers begin to usurp the decision-making role. If the only thing the AI did was collate information and present it to a human, nobody would be worried about it.
that's my pint about "where to look." You have given it data, it uses it to make a recommendation, which you can then use to dig deeper into teh solution to determine if it works. Ideally, it would tell you what logic it used to draw the conclusion so you can verify its correctness. One of the problems with using AI for say, stock analysis, is if enough people use the same logic it could become a self fulfilling prophecy as everyone buys or sell X at the same time, driving the price in the predicted direct
Re: (Score:2)
If a bot at a hedge fund gets caught profiting by insider information, what does the SEC do, pull its plug?
I suppose this also means that upcoming seasons of “Billions” won’t be nearly as good.
Re: (Score:2)
Right now most of the research documents are produced in India by people who are effectively little better than the AI you speak of; there is little insight and mostly just rote data. (This isn't because of the people doing it being Indian, but because the institutions don't value the data the same way they once did.)
Basically, every chance they get, big money goes for easier ways to profit. HFT was one of those things, but there is more.
p-hacking (Score:5, Insightful)
Re: (Score:3)
"With p-hacking, the larger the data set, the greater the probability of identifying background noise as significant patterns."
The root cause is, once again, "correlation does not mean causation". And stock traders are very keen to that fallacy: "I got five quarters in a row of profits for my customers, therefor you should trade with me". These systems are very good finding patterns *in the past*, does this mean those patterns will reproduce *in the future*? Heck, no, and the larger the data set, the more
Re: (Score:2)
There are correlations that are significant though. I remember when one of the first social network mining bots was used to predict stock price moves. We had seen it here on /. for several months at least-- plug for Corel led to a stock bump. It was great; you could easily double your money in a week.
(The real meaning though of those findings was that we were in the proverbial bubble where the bellboy is giving stock tips.)
Re: (Score:2)
Ah, it looks like the financial sector are going to explore the limitations of automated p-hacking.
This is a common and known problem with machine learning, where it is known as "overfitting". There are many remedies, the most important is to separate your data into "training data" and "testing (or validation) data".
Re: (Score:2)
I thought it was just my imagination seeing lions, mermaids, dogs and cats in the clouds. Once I even spotted Saraswati, Goddess of Learning and Knowledge in the morning toast and I aced the test on that day too. Coincidence? I think not. (BTW I am not Descartes, so I would not vanish)
Re: (Score:2)
While you can't trust those results, they give you good things to look at carefully, and eliminate a lot of things that wouldn't be useful.
This presumes that while you get a lot of false positives, you don't get many false negatives. You can usually set things up that way. When you do, the results are unreliable, but still useful. You just need to properly understand them.
Re: (Score:2)
This is a joke, right? obviously you did none of these things.
Re: (Score:2)
Sorry, the current level of AI won't but stability in the markets. What they're talking about here is basically trend detection, and that increases instability. To increase stability you need to focus on bubble detection, but nobody makes a lot of money that way. It can prevent loss of money, but the loss it prevents is "other people's money".
Give in (Score:1)
Eventually they'll get us all. [blogspot.com]
Comment removed (Score:3)
Re: (Score:2)
Re: (Score:2)
I don't know how the big houses do it, but one of the common tools is multi-dimensional spreadsheets which can let you easily run sensitivity analysis on various permutations. I wanted to get one of the programs to simplify a task I was doing, but they were simply too expensive given the core market they served. Ended up cobbling together a perl script to manage/mangle the data.
The problem is most of the research today is geared towards "slightly above/below average" companies. The exceptional companies
Re: (Score:2)
What are the exact analyses? (Score:2)
It would be more interesting to hear exactly what analyses they are, since it might give some fun ideas to the /. readership. But since it is a bit of a puff piece with a leading photo of a woman taking a selfie with a toy robot what do you expect? Guessing they have a clear training manual for their research analysts and they are able to automate 80% of that, then increase hiring of people who understand machine learning to improve that?
What pattern? (Score:1)
Re: (Score:2)
Re: (Score:2)
My question exactly. You can find lots of patterns in noise, but most of them will never repeat, and none of them will over the long run.
This once again goes to show ... (Score:2)
... that it's specialized upper end jobs that will be amoung the first to be replaced by AI/ML. Like, for example, ours. A buddy I did webdev with left the industry a few years ago and did an apprentice as a plumber for this exact reason. He can pick his employer.
Software development is being streamlined as we speak, and most of the work left to be done is mucking about with badly and very badly designed legacy systems and trying to migrate the whole shebang to something resembling a feasible concept runnin
Re: (Score:2)
Wow (Score:1)
Please, journalists, stop calling AI robots and robots AI.