Artificial intelligence is just smart\u2014and stupid\u2014enough to pervasively form price-fixing cartels in financial market conditions if left to their own devices.<\/p>\n
A working paper<\/a> posted this month on the National Bureau of Economic Research website from the Wharton School at the University of Pennsylvania and Hong Kong University of Science and Technology found when AI-powered trading agents were released into simulated markets, the bots colluded with one another, engaging in price fixing to make a collective profit.<\/p>\n In the study, researchers let bots loose in market models, essentially a computer program <\/a>designed to simulate real market conditions and train AI to interpret market-pricing data, with virtual market makers setting prices based on different variables in the model. These markets can have various levels of \u201cnoise,\u201d referring to the amount of conflicting information and price fluctuation in the various market contexts. While some bots were trained to behave like retail investors and others like hedge funds, in many cases, the machines engaged in \u201cpervasive\u201d price-fixing behaviors by collectively refusing to trade aggressively\u2014without being explicitly told to do so.<\/p>\n In one algorithmic model looking at price-trigger strategy, AI agents traded conservatively on signals until a large enough market swing triggered them to trade very aggressively. The bots, trained through reinforcement learning, were sophisticated enough to implicitly understand that widespread aggressive trading could create more market volatility.<\/p>\n In another model, AI bots had over-pruned biases and were trained to internalize that if any risky trade led to a negative outcome, they should not pursue that strategy again. The bots traded conservatively in a \u201cdogmatic\u201d manner, even when more aggressive trades were seen as more profitable, collectively acting in a way the study called \u201cartificial stupidity.\u201d<\/p>\n \u201cIn both mechanisms, they basically converge to this pattern where they are not acting aggressively, and in the long run, it\u2019s good for them,\u201d study co-author and Wharton finance professor Itay Goldstein told Fortune.<\/p>\n Financial regulators have long worked to address anti-competitive practices like collusion and price fixing in markets. But in retail, AI has taken the spotlight, particularly as legislators call on companies to address algorithmic pricing. For example, Sen. Ruben Gallego (D-Ariz.) called Delta\u2019s practice<\/a> of using AI to set individual airfare prices \u201cpredatory pricing,\u201d though the airline previously told Fortune its fares are \u201cpublicly filed and based solely on trip-related factors.\u201d<\/p>\n \u201cFor the [Securities and Exchange Commission] and those regulators in financial markets, their primary goal is to not only preserve this kind of stability, but also ensure competitiveness of the market and market efficiency,\u201d Winston Wei Dou, Wharton professor of finance and one of the study\u2019s authors, told Fortune.<\/p>\n With that in mind, Dou and two colleagues set out to identify how AI would behave in a financial market by putting trading agent bots into various simulated markets based on high or low levels of \u201cnoise.\u201d The bots ultimately earned \u201csupra-competitive profits\u201d by collectively and spontaneously deciding to avoid aggressive trading behaviors.<\/p>\n \u201cThey just believed sub-optimal trading behavior as optimal,\u201d Dou said. \u201cBut it turns out, if all the machines in the environment are trading in a \u2018sub-optimal\u2019 way, actually everyone can make profits because they don\u2019t want to take advantage of each other.\u201d<\/p>\n Simply put, the bots didn\u2019t question their conservative trading behaviors because they were all making money and therefore stopped engaging in competitive behaviors with one another, forming de-facto cartels.<\/p>\n Fears of AI in financial services<\/p>\n With the ability to increase consumer inclusion in financial markets and save investors time and money on advisory services, AI tools for financial services, like trading agent bots, have become increasingly appealing. Nearly one third of U.S. investors said they felt comfortable accepting financial planning advice from a generative AI-powered tool, according to a 2023 survey<\/a> from financial planning nonprofit CFP Board. A report<\/a> last week from cryptocurrency exchange MEXC found that among 78,000 Gen Z users, 67% of those traders activated at least one AI-powered trading bot in the previous fiscal quarter.<\/p>\n But for all their benefits, AI trading agents aren\u2019t without risks, according to Michael Clements, director of financial markets and community at the Government Accountability Office (GAO). Beyond<\/a> cybersecurity concerns and potentially biased decision-making<\/a>, these trading bots can have a real impact on markets.<\/p>\n \u201cA lot of AI models are trained on the same data,\u201d Clements told Fortune. \u201cIf there is consolidation within AI so there\u2019s only a few major providers of these platforms, you could get herding behavior\u2014that large numbers of individuals and entities are buying at the same time or selling at the same time, which can cause some price dislocations.\u201d\u00a0<\/p>\n