Forgot your password?
typodupeerror
Networking Software The Almighty Buck Hardware Technology

Undiscovered Country of HFT: FPGA JIT Ethernet Packet Assembly 452

Posted by timothy
from the fifo-with-a-vengeance dept.
michaelmalak writes "In a technique that reminds me of the just-in-time torpedo engineering of Star Trek VI: The Undiscovered Country, a company called Argon Design has "developed a high performance trading system" that puts an FPGA — and FPGA-based trading algorithms — right in the Ethernet switch. And it isn't just to cut down on switch/computer latency — they actually start assembling and sending out the start of an Ethernet packet simultaneously with receiving and decoding incoming price quotation Ethernet packets, and decide on the fly what to put in the outgoing buy/sell Ethernet packet. They call these techniques 'inline parsing' and 'pre-emption.'"
This discussion has been archived. No new comments can be posted.

Undiscovered Country of HFT: FPGA JIT Ethernet Packet Assembly

Comments Filter:
  • Wow, (Score:3, Insightful)

    by Antony T Curtis (89990) on Thursday September 26, 2013 @12:16PM (#44961187) Homepage Journal

    This is the madness of high-speed trading...

    • Re:Wow, (Score:4, Insightful)

      by Anonymous Coward on Thursday September 26, 2013 @12:30PM (#44961327)

      High speed trading is all about stealing as much money as quickly as possible before anybody else has a chance to do it.

      It serves no purpose but to move money from the hands of everyone else to the banks -- the same banks that caused the financial mess in the first place.

      I blame this squarely on the American concept of capitalism -- the Republicans and Libertarians will tell you this is how it's supposed to work, because they and their benefactors directly profit from this.

      The super rich just skim off the top and bypass the entire 'market' for their own purposes.

      And this is why there never has been, and never will be a free market and it can't do what it claims -- because someone will always game the system for their own benefit, and there's nothing to keep them in check. And the big players define the market choices, so there is no actual options for people.

      I view HFT as wholesale theft, which is pretty much exactly what it is. Nobody earned anything, they just have a direct hook into the system whereby they can bypass everything and make money before it exists to anybody but them.

      Burn Wall Street. Eat the rich.

      • Re: (Score:2, Informative)

        by Anonymous Coward

        That's right!
        I understand that none of the Democrats will have anything to do with WallStreet.
        Heck, not a single one of them owns any stocks from what I hear.

      • Re:Wow, (Score:5, Insightful)

        by WheezyJoe (1168567) <fegg@exc[ ].com ['ite' in gap]> on Thursday September 26, 2013 @01:40PM (#44962267)

        Burn Wall Street. Eat the rich.

        No, tax Wall Street, Tax the rich.

        Why? To punish the rich for being rich? To punish Wall Street for making so much money?
        No. To control the market, to put a disincentive to trading for trade's sake and locking up capital in the hands of a few until they come to control the market itself.
        After the last Great Depression, people realized that Wall Street will always go out of control, eventually. We've forgotten those lessons, and removed the regulations that kept Wall Street boring (it sure wasn't sexy to be a banker in the 70's) and kept capital where it belongs: industry.

        Under those old rules, Wall Street made money by performing financial services, and thus sharing in the success of industry, but otherwise sat on the sidelines.

        Trading for trading's sake, however, is simply buying low and selling high. No matter how complex the instrument, it all boils down to buy low, sell high. And therein lies the rub; somebody has to lose.

        Yep, there has to be a loser. One of the parties in the trade is going to get something that's worth less than they think it is, and the other's going to profit from it. Arbitrage, for example, is where some poor fool is offering more money to buy something than some other poor fool is selling it for. Wall Street swoops in, buys from the one and sells to the other, banks the difference. Free money! Too bad the two fools couldn't have found each other on their own... both paid/received the wrong price.

        HFT only facilitates one bank making the deal (getting at the sucker) quicker than another. And the higher the speed, the less thinking the other party has to consider whether the deal is any good or not. Assuming that the first party can think that fast, which they probably can't. But they can't appear to be trading with yesterday's tech, can they? Otherwise, people will consider them out-dated, and target them for a sucker.

        Buy low, sell high. I know this stack of Mortgage Backed Securities is worthless, but you haven't heard the news and I'll tell you it's great and you have 15 seconds to make the trade or it's going to someone else. That's right, little fish, take the bait.

    • by gmuslera (3436)
      The madness goes above it. In a system where the ultimate good is money, going all for it is a reasonable move. Making dedicate machines that make money as fast as possible is a good approach. The madness there is that the ultimate good is money (for everyone, even the ones that make the laws), more than human life or that mankind or civilization have a future.
    • Rewards (Score:4, Insightful)

      by Spazmania (174582) on Thursday September 26, 2013 @02:07PM (#44962697) Homepage

      We need to stop rewarding folks for high-speed trading. It basically steals money from the folks genuinely invested in the companies whose stock is traded and adds no value to the system at all.

  • What a waste (Score:2, Insightful)

    by stewsters (1406737)
    These people could be used to develop more efficient hardware for everyone to use, or fix medical conditions, rather than make rich traders even richer at the expense of another economic collapse. It seems wrong that our economy prioritizes high frequency trading so much.
    • by larry bagina (561269) on Thursday September 26, 2013 @12:28PM (#44961309) Journal
      You could be used to develop more efficient hardware for everyone to use, or fix medical conditions, rather than posting on slashdot.
    • by internerdj (1319281) on Thursday September 26, 2013 @12:34PM (#44961379)
      Apparently they make life possible because they move capital where it is needs to be, and deserve the medal of honor for what they graciously give the rest of us. http://finance.yahoo.com/blogs/daily-ticker/99-owe-debt-gratitude-1-harry-binswanger-153327379.html [yahoo.com]
    • It seems wrong, because it reflects a fundamental lack of concern for what securities represent. It is another step on the road of investment being more valuable than insight, labor, or creating meaningful value.

      The underlying purpose of the stock market is the valuation of companies and what they do, but we see more and more evidence that the metadata outweighing the data. It's a subtle incrimination of corporate capitalism, that those involved really don't want to acknowledge.

      • You can have a room full of insightful, creative people. But without investment, they won't be accomplishing much.

        • by sjames (1099)

          OTOH, HST is not investment at all. Th market has become so messed up that it's easy to get investment for mail order pet food, but not for useful technology.

          • That has nothing to do with the market and everything to do with the fact that the US is a consumer driven society.

            Selling pet food brings an immediate, strong and much more certain return on capital than developing a new technology.

        • by curunir (98273) *

          That room full of insightful, creative people gets their money from venture capitalists/angel investors. The only role the stock market has is making it easier for venture capitalists to exit from successful investments. Without the stock market, there could still be investment made in early stage companies, but investors would have be more discerning about what they invest in because they'd have to make their money back through company stock buy backs, dividends or acquisitions.

          The way that investment happ

        • by GameboyRMH (1153867) <gameboyrmh@[ ]il.com ['gma' in gap]> on Thursday September 26, 2013 @01:30PM (#44962101) Journal

          So thank goodness their moneyed overlords are willing to lend some of the cash made on the backs of such creative people, so that the creative insightful people can do more work and give the lion's share back to the moneyed overlords. It's a good thing we have a dynastic class of people controlling all the capital.

      • Effective allocation of capital is absolutely critical to an efficient economy. It is a sine qua non of free societies.

        It is why centrally planned economies don't work in competition with market economies. Look what happened to China when they released some the the reins vs. Mao's 5 year plans.

        Why do you think the credit collapse had such a dire effect on the world economy 5 years ago?

        Labor and creativity are important too, but they don't turn into meaningful economic activity without capital.

        Investment is

        • Re:What a waste (Score:4, Insightful)

          by Urza9814 (883915) on Thursday September 26, 2013 @01:25PM (#44962031)

          High frequency trading isn't investment, it's gambling. It's hard to do any "meaningful economic activity" when the amount of money you have to do it with changes every couple milliseconds....

          Investments are long-term. Investments are saying 'here's some money, you go build X and give me Y% of your profits'. That's an investment. And you don't need millisecond speed trades and specialized FPGA NICs if you're leaving your money there long enough for R&D, marketing, and sales. Investments last months or years, not minutes or milliseconds.

    • by Alioth (221270)

      Maybe so or maybe not. A bit like the arguments "We shouldn't have both Gnome *and* KDE! Everyone should work on just one project so we get more done". The thing is maybe people working on HFT systems don't want to work on medical devices and wouldn't work on medical devices even if HFT didn't exist. I don't happen to work on HFT systems myself, but personally I wouldn't want to work with the stifling regulations there are for medical devices, I'd rather work on something that's rather more free in how you

      • by h4rr4r (612664)

        You think they do it for the love of HFT?

        For that kind of money I would gladly deal with medical device regulators all day.

        • There are some pretty interesting technical challenges with HFT and a competitive problem that isn't inherent to most technical problems which tend to often be man versus nature. It could have some appeal. That said, there isn't an amount of money and set of technical challenges interesting enough to make me want to work for that type of management structure and corporate environment.
          • by Urza9814 (883915)

            Yeah, interesting challenges like properly timing your insider trades to not appear to defy the laws of physics ;)

    • Re: (Score:3, Interesting)

      by Anonymous Coward

      They have. See https://en.wikipedia.org/wiki/Alphamosaic for their previous jobs. Also, 3rd generation of their chips is in the Raspberry Pi, as the CPU.

    • by gandhi_2 (1108023)

      Unlike the noble persuit of high-end graphics cards to build a totally bitchin' mining rig.

      • Those high-end graphics cards have some more noble uses too.

        MRI scanners, for a start. It takes a huge amount of processing power to turn a tangle of radio signals into a rendered image of the patient. The early machines took hours to produce an image - with optimised hardware today, it's possible to generate them almost in real time mid-surgery.

    • by Bengie (1121981)
      I see it like racing. Lots of money goes into making fast cars go faster for something that has no real value. Yet many of the technologies that go into race cars still make their way into regular cars.

      These kind of situations making crazy ideas worth trying and lots of money to back it. How long until home users get the benefit for FPGA assisted networking because it becomes a well-know subject and gets crazy cheap to implement because some rich people wanted to try to make even more money in trading?
      • The big difference is that racing doesn't skim money by acting as a middleman on other people's gas purchases. If HFT were taking place in a safe, closed environment as a sport it would be a vast improvement.

    • by nurb432 (527695)

      Without wealthy people you wont get your research for 'everyone else to use'.

  • by aepervius (535155) on Thursday September 26, 2013 @12:17PM (#44961203)
    So that you can send your order before even the price quotation comes. Oh , wait, those guy already can do that and send packet back in time : http://news.slashdot.org/story/13/09/25/1955220/somebody-stole-7-milliseconds-from-the-federal-reserve [slashdot.org] /sarcasm
    • by necro81 (917438)
      There has been speculation of people developing neutrino-based communication systems for high-speed trading. Because neutrinos pass through matter more-or-less unabsorbed and unimpeded, and travel at more-or-less the speed of light, you can do line-of-sight communication straight through the Earth. So, for instance, a trader in NY can send buy/sell orders to Japan by beaming neutrinos through the Earth and have them arrive faster than orders sent via optical fiber over the surface of the Earth (which is a
      • Neutrinos? Bah. A wormhole lets you shrink the distance between 2 points, so the speed of light is less of a problem.

      • by Minwee (522556)

        So, for instance, a trader in NY can send buy/sell orders to Japan by beaming neutrinos through the Earth and have them arrive faster than orders sent via optical fiber over the surface of the Earth (which is a longer path, and the speed of light in optical fiber is less than c in a vacuum), or via satellite.

        But even if they were using tachyons for communication, their orders would never arrive before the ones which come from a rack that is only five metres away from the exchange and connected directly to it with a straight optical cable. Even if the trader is sitting in NYC, they can still run their algos in Tokyo and get the jump on any other traders who don't.

        Why resort to science fiction when there is already a better way?

        • Because there is more than one exchange. New York, London and Tokyo run the big ones. If you can get faster inter-exchange communication than anyone else, you can exploit the small differences in prices between them.

          eg: Someone buys a ton of stock in Widgets, Inc in New York. This causes the price of that stock to suddenly rise, in a matter of miliseconds. Information reporting this is sent out as fast as the fiber will carry it towards London. Meanwhile, your sneaky neutrino communicator sends out the news

  • by hirschma (187820) on Thursday September 26, 2013 @12:17PM (#44961205)

    Things like high frequency trading make me want to vom. Essentially, all they're doing is shuffling money around, taking advantage of an outdated system, and increasing risk for the entire world.

    It'd be great to see this kind of innovation in something that actually is useful and valuable - not for creating an incremental improvement on a corrupt system.

    • by KiloByte (825081) on Thursday September 26, 2013 @12:22PM (#44961249)

      Essentially, all they're doing is shuffling money around.

      More exactly: they're shuffling money from our accounts to theirs. If your bank has any securities involved with the stock exchange, it is your money that is getting nicked here. A zero-sum game requires that besides a winner there must be a loser.

      • by HornWumpus (783565) on Thursday September 26, 2013 @12:32PM (#44961341)

        The big loser is the trading clearinghouse/broker. Bid/ask spreads are about 1% of what they used to be.

        The other big losers are speculators (as opposed to investors, look up the difference).

        I see no problem with this except when the HFtraders are able to say: 'Our bad, back out all those losing trades for us'. That's just fucked.

        • Except HFT is causing bigger spikes on stock prices, helping the speculators and hurting the investors who decide to get in or out at the wrong time.

          • There is no evidence of that. All the really big stock price events occurred in the past. Like the 22% one day drop in 1987. In fact it's likely HFT reduces large spikes because the arbitrage occurs much more quickly.

            And also just so you know, investors don't try to time markets. They hold for long periods of time so these fluctuations don't matter.

          • by Bengie (1121981)
            If you an investor buying during a spike, you're doing it wrong.
        • by AmiMoJo (196126) *

          Just wait until someone figures out how to write an exploit targeting HF trading systems. Within milliseconds they could execute the biggest cyber attack in history and crash the economy of an entire country. They could even interlock it with trades from other countries just to make un-picking all the damage nearly impossible.

          Of course it might just happen that way due to a genuine bug in one of the systems. Would be hard to tell which it was from a distance.

          • This would be really hard to do. Similar things have happened by accident, at least within a single stock exchange, and the trades are simply rolled back.

        • Your basic knowledge of trading has no place here.

          Keep in mind that you just replied to someone who thinks copyright laws are a "crime against humanity".

      • Re: (Score:3, Insightful)

        I think most non-finance people who read a little about high-frequency trading systems are repulsed by the concept. Can someone point out a useful economic function for them?

        One the one hand they dramatically reduce the bid-ask spread for low-volume trades. But is that really all that economically beneficial?

        On the other hand, they prey on large trades that mutual funds must place to rebalance their holdings. This probably hurts the average investor quite a lot.

        Some claim that they increase liquidity and re

    • by gmuslera (3436)
      You are playing their game too, you just get outraged because they play it more efficiently. They played enough with the rules to make millons just moving aluminum between 2 warehouses [nytimes.com]. Life is more than money, but for our current culture is everything.
    • by lewkor (111443)

      I fail to see what possible connection this has to reality. I could dig up my old economics text book and trot out my old economics theory, but I don't have to. What possible change of the value of a good or service can take place in time scales of less than a milisecond? Computer trading systems are a fraud perpetrated by people that have a severe need of a reality check!!!

      As the parent post alluded to all this serves is to make a dynamic system less stable. If the actions super rich (who else uses the

  • Fuck off and die (Score:2, Insightful)

    by Anonymous Coward

    Fucking parasites (and their toolmakers).

  • by Anonymous Coward

    Makes no sense.

  • by Anonymous Coward on Thursday September 26, 2013 @12:25PM (#44961279)

    I have a friend who works in the defense industry but interviewed with HFT firms around 3 years ago. My friend also had this idea, and discussed it with me since I am close to the industry.

    If an industry outsider like my friend had this idea within a week or two of merely interviewing for jobs, it is a good bet many others had already conceived it and even gotten it working before that.

  • by b4upoo (166390) on Thursday September 26, 2013 @12:32PM (#44961351)

    Just think of the expenses that could be involved if one of these programs screws up. Instead of sell! sell! sell! One could buy! buy! buy! the wrong stock in very large quantities. Instead of a billion dollars in buggy whips one might end up with a billion dollars in donkey whips. What a bummer.

  • by XanC (644172) on Thursday September 26, 2013 @12:45PM (#44961485)

    What is with the vitriol here? Why should buyers and sellers not be able to come together to make a transaction at any time that they like?

    • by SirGarlon (845873)

      The point of high-frequency trading [wikipedia.org] is for the buyer to pull a fast one: to offer a price that an informed seller would not accept, before the seller can find out what the fair price should be.

      • Re:Don't understand (Score:5, Informative)

        by GameboyRMH (1153867) <gameboyrmh@[ ]il.com ['gma' in gap]> on Thursday September 26, 2013 @01:48PM (#44962411) Journal

        Exactly. Say there's a classified site that you can only load once per minute due to bandwidth restrictions (being a human). I post "Bicycle for sale $500" and another guy posts "wanted: bicycle, $600 or less."

        But there are some guys who can reload the page faster because they've bought a very expensive premium service from the classified site. Not a very fair site is it?

        One of them sees the two ads, buys my bicycle, and posts "Bicycle for sale $599" before any non-premium members can see what's going on.

        Who did that help except for the guy with the premium service? I didn't make more. The guy who wanted a bike just got screwed out of $99.

    • by pla (258480)
      What is with the vitriol here? Why should buyers and sellers not be able to come together to make a transaction at any time that they like?

      They should!

      But computers can't "like" ice cream, either.
    • by sjames (1099)

      Because this is rent seeking and has a lot more to do with jumping in between legitimate buyers and sellers to extract money than it does with investment.

      Also because the people doing it have already managed to crash the world economy once and through political maneuvering got a handsome payout for doing it rather than the jail time they deserved. Because in spite of the big handouts from the taxpayers they continue to shit all over everyone.

      They should be roasted on a spit until they stop screaming.

  • Actual full summary ending: "They call these techniques 'inline parsing' and 'pre-emption' and 'greedy unfair asshole companies making billions for their owners through cheap, cheating tactics like this to undercut smaller startups and push them out of the market so their fat owners can buy another personal jet instead of handing off the profits to their investors/customers'"
  • by wonkey_monkey (2592601) on Thursday September 26, 2013 @12:52PM (#44961585) Homepage

    In The Undiscovered Country they modified a torpedo to home in on gas emissions from a Klingon Bird of Prey. This is a story about building trading algorithms into an ethernet switch.

    Apart from "needing to do something quickly," I really, really can't see the connection.

  • Making lots of money by pushing network packets around faster, to no real net benefit to anyone. Other than the ones pushing the packets.

  • They call these techniques 'inline parsing' and 'pre-emption.'"

    And everyone else has been calling this "pipelining" for decades.

    • And everyone else has been calling this "pipelining" for decades.

      I'm not aware of any pipelining system that performs partial parsing. In CPU pipelining, "instruction decode" is AFAIK a single, atomic step.

      • I'm not aware of any pipelining system that performs partial parsing.

        As far as I'm aware, TeX has been processing its input in this way since time immemorial. Pipelining is a generic concept, it's not constrained to CPU instruction processing.

  • I think we just found our culprit.

  • Having a pick set either declares you as an intruder, or someone who loses their keys more often than the pick set...

    HFT is morally thievery, ethically abusive, and deserves to be regulated as such.

    OR, alternatively, brokers, market makers, and exchanges need to fully and repeatedly disclose to investors the nature and impact of HFT. Those of us trying to time the market are wasting our time, the HFT guys have this down to milliseconds. Trying to find arbitrage is impossible.

    Of course, when the Fed stops

  • by Tom (822) on Thursday September 26, 2013 @02:37PM (#44963127) Homepage Journal

    Some things in life are simple:

    Almost everyone agrees that HFT is evil.

    Nothing is being done about it.

    How can both of these things be true at the same time without revealing a serious, dangerous flaw in our political and economic system?

Men love to wonder, and that is the seed of science.

Working...