Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×
Transportation Data Storage Technology

San Francisco Muni's Rail System Will Spend $212 Million To Upgrade From Floppy Disks (govtech.com) 96

San Francisco's Municipal Transportation Agency approved a $212 million contract with Hitachi Rail to modernize the Muni Metro system's outdated train control system, which currently uses floppy disks and wire loops. Government Technology reports: The software that runs the system is stored on floppy disks that are loaded each morning and an outdated type of communication using wire loops that are easily disrupted. It was expected to last for 20 to 25 years, according to Muni officials. It moves data more slowly than a wireless modem, they said. By late 2027 and into 2028, a new communications-based system, which employs Wi-Fi and cell signals to precisely track the locations of trains, will be installed by Hitachi, which will provide support services for 20 years under the agreement.

While the current train control system operates only on the Market Street subway and Central Subway, the new system will control Metro light rail trains on the system's surface lines as well. The Hitachi system is said to be five generations ahead of the current system, said Muni Director of Transit Julie Kirschbaum, who described it as the best train control system on the market.

This discussion has been archived. No new comments can be posted.

San Francisco Muni's Rail System Will Spend $212 Million To Upgrade From Floppy Disks

Comments Filter:
  • by Anonymous Coward
    I could upgrade them to LS-120 for a fraction of the cost.
  • After a few times needing to walk through a portion of the tunnels as the system failed, they had finally upgraded the system to modern communications that would allow for driverless trains... just a "safety driver." Interesting that the solution was neither modular or easily upgraded as the system grew over the next few decades.

    • by rta ( 559125 )

      Heh. from the TFA it looks like $212M is just the communications part or some subset of the system:

      The contract is part of a $700 million project to transform the Muni Metro's control system.

      Also i don't see how they're going to use ATC outside even now when something like half the system is at street level and interacts with pedestrians, traffic lights and car traffic.

      • depends if they have a coms network / radio spectrum allocation

        other countries railways have a portion of the spectrum they can run their own LTE network on not sure if Muni can do that...

        if they could it would be cheaper to use GSM technology with voice for the drivers etc etc

        instead they will go with the old CBTC and live on things like Wifi and hope interference does not kill them...

        good luck

      • They use drivers above ground.

      • by arglebargle_xiv ( 2212710 ) on Wednesday October 23, 2024 @03:18AM (#64886531)
        Fast-forward to Slashdot, 2029:

        After going 300% over budget and several years behind schedule, the MTA have abandoned plans to use Hitachi's system, which had caused extensive transport outages and problems for passengers, and gone back to using floppy disks. The MTA is suing Hitachi for breach of contract, and Hitachi is countersuing alleging the MTA's constantly-changing requirements were the cause of the problem".

        • Mod this up ! ^^^

          And mod "Informative". It would be funny if it wasn't for the number of projects where we see *exactly* this sort of bullshit.

        • by flug ( 589009 )

          And the 2029 update continues:

          The 1998 floppy-disk driven system continues to operate as designed. Muni obtains needed floppy disks from the one remaining supplier on eBay, who continues to gradually sell his remaining stock - currently at $199.95 per disk. He estimates 10 years floppy disk stock remaining, by which time the price is predicted to reach $1999.95 or the equivalent, one bitcoin.

  • Remember! (Score:5, Funny)

    by haxor.dk ( 463614 ) on Tuesday October 22, 2024 @09:22PM (#64886123)

    Don't copy that floppy!

  • Do they mean 4-20 ma systems? Modbus? Something really old?

    Isn't there something like this for their system?

    https://www.bigmessowires.com/... [bigmessowires.com]

    Granted it probably is time to rip it all out and start over. I helped do something similar at a chemical plant, but we had a 10 day maintenance shutdown to do it.

    • I believe they're speaking of induction detectors [wikipedia.org] to detect when the train passes.
    • Start over - can't (Score:5, Insightful)

      by FeelGood314 ( 2516288 ) on Tuesday October 22, 2024 @09:49PM (#64886149)
      They don't know what the current system does. I've worked on a number of these legacy systems and they have had crap slowly added to them over time and all the documentation is now lost, out of date or for a completely different system. It is unlikely that you could find a quarter of the source code of what is running. And the source code you would find would be full of #ifdefs that were likely set as compile options but no one knows which options the current image has. You would need a team of competent engineers to bang their heads at it, cry and pull their hair out for months just to get a high level idea of what the parts are. 6 months to even get an estimate of what needs to be done, again assume really good engineers. Then you would need a good manager who would break the large system up and have small parts reimplemented and tested. Oh, you likely need to test on the real system to actually know if you got things right and didn't break something*. It would take a miracle for that to be allowed. Then maybe, going piece by piece a small team of 5 or 6 engineers could replace the entire thing in 2 years. 2 years, 3 million dollars and all you would have to show is the same functionality.

      *Maybe someone bright could build you an emulator of the system for a couple million before you even start?
      • " I've worked on a number of these legacy systems and they have had crap slowly added to them over time and all the documentation is now lost, out of date or for a completely different system."

        I've been there, LOL. DEC Alpha running sequences. It was still going in 2013 when they shut the whole unit down. Then there was 1985's finest Provox system feeding it data.

      • the voice of experience
    • TFA says the system is from 1998 and already "the writing was on the wall" that floppy disk technology was in it's sunset phase. But this "going around inserting floppies every morning" really makes no sense. Flash memory was expensive back then but hardly a drop in the bucket when you are spending millions on a system like this. Hard drives were an option too (spin up, load the system software, spin down until next morning). Unless there is some details missing here?
    • >Do they mean 4-20 ma systems? Modbus? Something really old?

      That system is still very, very much in use.

  • "The software that runs the system is stored on floppy disks that are loaded each morning" because desktop hard drives or network booting did not exist? How old is their equipment? :-|
    • This could have been a security/safety feature. If the machines are unattended at night, who knows what shape they are in the next day. By taking the floppy disk out at night and rebooting in the morning, you reduce the risk of casual tampering. Or it might just have been silly.
      • Long, long ago I had a friend with a PC that had two floppy disks and no hard drive. He could copy anything and did not have to fear viruses (DOS was already loaded into memory).

        Maybe this is part of the problem with replacing floppies?

        • If that's the *only* issue, botting from a read-only USB stick would achieve the same thing and more. I imagine that there are also other issues moving from the floppy disks. But the ability to get back to a known state every morning seems to be part of the design of the system and a smart requirement. Replacing a floppy disk is something anybody could do and could be handled by the same people who unlock the gates. Opening the machine and replacing a failed hard drive and then reinstalling the OS/softw
    • by necro81 ( 917438 )
      Well, the article indicates that the current floppy-based system "was expected to last for 20 to 25 years." Alas, hewing to true modern journalistic standards, they didn't bother to mention how old the system actually is. But since they added that detail, presumably it is now substantially outside its original life expectancy. So call it 30 years old: deployed in the mid-90s, using a design and technology from the early 90s or late 80s.

      They didn't specify just what kind of floppy they're talking about
      • Re:Huh? (Score:5, Informative)

        by timeOday ( 582209 ) on Wednesday October 23, 2024 @08:45AM (#64887067)
        This article from April of this year has those details:

        Officials from the San Francisco Municipal Transportation Agency (SFMTA) told KGO-TV in a recent interview that 5.25-inch floppy disks are still used to operate the city's train control system.

        The outlet reported the system was built to last up to 25 years, and now that it is a year past, officials fear it could break down.

        https://www.foxbusiness.com/te... [foxbusiness.com]

    • > because desktop hard drives or network booting did not exist?

      HDDS's were tiny and expensive.

      Network? What network?

      You might wonder how all the planes stay up in the air too, many of those still use floppy based updates and they aint going away very soon.

  • This will fail (Score:5, Insightful)

    by gweihir ( 88907 ) on Tuesday October 22, 2024 @09:55PM (#64886159)

    My guess is that this will fail because there is too much money involved. Just an itutition after having seen too much IT disasters.

    Let's revisit this prediction in 10 years.

    • The reason floppies are still used in nuclear weapons silos and trains and Boeing planes is their simplicity and robustness to degradation. Making it more complex does not itself make it better just be cause its newer.

      I don't think even 10 years is needed, I expect it will be hacked or have a technical meltdown within 5. We shall see.

      • Nearly all the floppies i still have from the 1980s and 1990s have developed bad sectors. They are not resistant to degradation.
        For durable media, my DAT/DDS tapes from the same era still work fine. However they are sequential access only and probably not suitable for the application in TFA.

      • Also cost.

        It cost several million to install the original system. To replace it you need to spend many new millions more.

        Vs the cost of sourcing, maintaining what you have, including the 8" floppies, the slight inconvenience vs the millions needed doesnt equate.

        Eventually it will. Unless they can siply save money by commisioning 3rd parties to make new parts etc, which again may be cheaper.

        Eventually you might end up with a "Triggers broom" situation, where the original system is largley replaced with new

        • by gweihir ( 88907 )

          If they are smart, they will not go for a fundamental replacement. They will just modernize some parts. For example, there are nice hardware and software emulators for floppies and those eliminate any ageing problem the real hardware may have. But the sum they plan to spend tells me they are trying a complete replacement, and I highly doubt they will succeed in that. More likely, the money will be gone at the end and nothing will have been accomplished.

  • by Hey_Jude_Jesus ( 3442653 ) on Tuesday October 22, 2024 @10:03PM (#64886171)
    Isn't progress wonderful? ;-)
  • Interim solution (Score:5, Insightful)

    by Rosco P. Coltrane ( 209368 ) on Tuesday October 22, 2024 @10:12PM (#64886191)

    Hit the retrocomputing community: I bet they could easily keep the current system as-is for the time being but upgrade the floppy drives to flash memory or even make them look like physical drives to the original system but pull files on the network on the other, with some el-cheapo Raspberry Pi board or something. I'm always amazed to see what these guys come up with to keep truly ancient computers going and relevant today.

    • For the Amiga we use goteks. https://www.retropassion.co.uk... [retropassion.co.uk] I find it hard to believe they couldn't use something similar.
    • We would just relpace the floppy discs with gotek drives, then you can happily use usb flash drives.

      I think what they are after however is to move away from the act of having to load the files into the system at all, automate it and then watch the whole thing crash one day leaving stranded passengers etc and greybeards remembering the old system where it simply couldnt happen :D

  • by pencil_ethics ( 1265384 ) on Tuesday October 22, 2024 @10:37PM (#64886229)
    Personally I think that the fact that floppy disks are still being used receives unnecessary focus (per the SFMTA's own words, this is 1980s tech installed in 1998, what do you expect). The more salient point is that (per https://www.sfmta.com/projects... [sfmta.com]) they are upgrading the existing train control system from the older ATCS (subway only) and manual control (everywhere else, and when ATCS comms fails) to CBTC - this allows the control center to know much more accurately where trains are at any given point in time. This article from 2019 (https://medium.com/@SFTRU/muni-metro-train-control-challenges-5985b0e4912c) is a good overview of what the issues are; the author clearly got his wish, because CBTC is exactly what is happening (-:
    • 1980s ATCS: "floppies, failing equipment with no new parts being made for it, and laughably out of date, OH MY!" 2020s ATCS: "That will be $8,000 m/o for automatic emergency braking. For only $3,000 more you can upgrade to the "Pro" version..." I hope I'm kidding.
  • $212 million (Score:3, Interesting)

    by backslashdot ( 95548 ) on Tuesday October 22, 2024 @10:41PM (#64886239)

    Are they that fucking stupid? How can it cost $212 million? Do these morons not have anyone that knows how computers work?

    • For real. Slap some air tags on the carriage and call it a day. Your welcome.

      Being facetious of course but one could think of dozens of ways to track trains and use that info to route, re-route, prevent accidents, etc for a mere fraction of 212 million.

      And yeah, going back to the original vendor after 25 yearsâ¦that Hitachi sales guy has retired on this bokndoggle

      • They already do some GPS-based train tracking (cf. https://www.sfmta.com/train-co... [sfmta.com]). They are spending what they are to install CBTC, which goes above and beyond just tracking trains. (https://en.m.wikipedia.org/wiki/Communications-based_train_control)
        • I read the Wikipedia and another article written by Muni. Standing by the assertion that $212 million is way too much for train tracking and routing over the entire system.

          Hire 6 retirees from JPL and give them a million each. Done in 12 months and rock solid for a decade to come.

          • Re: $212 million (Score:4, Informative)

            by pencil_ethics ( 1265384 ) on Tuesday October 22, 2024 @11:29PM (#64886303)
            Then you are woefully underestimating the scale of what is involved here. At a bare minimum, you have to retrofit all of the existing trains and tracks with new hardware (getting rid of the old stuff in the process), upgrade the control center systems, not to mention training, testing and compliance. This is not a 12 month/$6m effort.
          • by DrXym ( 126579 )

            Reality would beg to differ.

            • by wangi ( 16741 )

              The reality is that this work is being done in a $212M contract.

              Had you challenged that reality, responded to the tender with your $6M wonder plan, then I doubt you'd be sitting posting on the /.

              • by DrXym ( 126579 )

                My wonder plan? Where did I suggest it?

              • The basis of the argument above is that because the government (City of SF no less) agreed to pay $212 million, that's what it should cost.? They paid ~$2 million for a park toilet, is that what a toilet costs?

                Reality is that Muni Metro is very, very small
                ~180 carriages
                ~70 miles of track
                ~117 stations/stops

                The tech for tracking be it GPS/Cell triangulation, LoRa, whatever is very inexpensive. Underground is only a slight variation. The vaunted "CBTC" in addition to tracking, and spacing trains also does

                • by DrXym ( 126579 )

                  I suggest you look up the cost of other such projects to other metro & light rail systems, e.g. "CBTC rollout cost". This involves replacing all of the onboard, trackside and control room hardware & software, planning & installing new PLCs, sensors and signals onboard & trackside, programming every single PLC, testing and certifying its operation. So yes it's programmatically difficult.

                  And safety isn't about "hardening" a system either. Every single aspect of safety has to be considered in t

                  • by Pascoea ( 968200 )
                    It cracks me up how people can look at something like this, mentally reference something they whipped up on an arduino in their home lab, and think that extrapolates out to real-world applications. Yeah, tracking a train is easy. Controlling a train is straight forward. Coordinating trains, in concept, isn't exactly rocket surgery. Doing those safely and reliably for 20 years is mind-bogglingly complicated, and expensive. If it were easy, there would be thousands of companies out there doing it. There's
    • no. no one knows how computers work. its a collective effort of thousand of people around the world keeping a complex ever evolving stack of jenga tower bricks from toppling over.

      you could probably kill 10 people and setback the entire industry decades.

      • by mu22le ( 766735 )

        you could probably kill 10 people and setback the entire industry decades.

        Also, you could probably kill 10 people and force the entire industry to move forward by decades

    • Floppy disks are just one part of the grand scheme of things; the point is that they're upgrading the train control system (cf. my above comment), and getting rid of floppy disks is an added bonus
    • Re:$212 million (Score:5, Interesting)

      by DrXym ( 126579 ) on Wednesday October 23, 2024 @04:23AM (#64886593)

      I imagine that price includes a lot of trackside industrial control hardware, sensors, wifi / cell comms & power wiring along the length of the tracks, servers, an entire control room, developing software for the same, upgrades to junctions, lights & other signalling, upgrades to onboard cabins / safety systems / brakes / controls, safety certification, training and a bunch of other stuff. Plus support.

    • Re:$212 million (Score:5, Insightful)

      by hackertourist ( 2202674 ) on Wednesday October 23, 2024 @05:02AM (#64886637)

      It's costing $212M to replace the train control system. A side effect of that is that the floppy disks are phased out.

      • Also from the summary,

        a new communications-based system, which employs Wi-Fi and cell signals to precisely track the locations of trains, will be installed by Hitachi, which will provide support services for 20 years under the agreement.

        I am assuming that 20 years of support is included in the price tag.

    • Are they that fucking stupid? How can it cost $212 million? Do these morons not have anyone that knows how computers work?

      Of course they do. Hell, I’ll bet the first group of “moron” consultants were turned down because they quoted a paltry $175 million.

      You’d have to be that fucking stupid to believe this is gonna come in under budget too. Here’s a summary of how things are going in the transport department of California corrections:

      California’s high-speed rail project is infamous for delays and cost overruns since it was first pitched to California voters in 2008 as a $30 billion project that would be complete by 2020. Officials have since updated its price tag to $100 billion and estimate its first leg will be open to the public in 2033.

    • Transport for London paid £350m ($570m) in 2011 to upgrade the signaling system on their subsurface lines.

    • by mjwx ( 966435 )

      Are they that fucking stupid? How can it cost $212 million? Do these morons not have anyone that knows how computers work?

      That's precisely the problem. They've farmed the work out to the likes of Infosys who have no fucking idea how computers work... only how to provide a very large number that is slightly smaller than the one someone who knows how computers work will provide.

    • by Pascoea ( 968200 )
      Genuine question: What do you think it should cost? And what, exactly, is in scope for your guess?
      • >And what, exactly, is in scope for your guess?

        Exactly this.

        Quoting a cost of "X", state exactly what the statement of work was. Otherwise, people are just cherry picking for clicks

      • $10 million delivered 12-14 months
        -Real time tracking for each carriage
        -Train spacing (for safety)
        -Routing and control (i.e. routing around failures, special schedules, etc)
        -Redundancy
        -Safety and testing
        -Training
        -1 year support

        SF Muni Metro
        -------------------
        ~180 carriages
        ~70 miles of track
        ~117 stations/stops

        • by Pascoea ( 968200 )

          $10 million delivered 12-14 months

          Snork. Thanks for that, I haven't had a good laugh in a while. Oh, wait, you're serious?

          I read some of your other comments on the topic. specifically this one [slashdot.org] I'm sorry, but you don't have a clue what safety critical controls cost. Sticking an arduino with a LoRa radio and GPS receiver in each car and command-and-control system running on a server in the closet isn't going to cut the mustard. Plus you have to install the whole thing on a train system that will be in service the whole time. Hell, the cost to

          • Sticking an arduino with a LoRa radio and GPS receiver in each car and command-and-control system running on a server in the closet isn't going to cut the mustard.

            What do you think Hitachi is going to do?

  • by Uldis Segliņš ( 4468089 ) on Tuesday October 22, 2024 @10:59PM (#64886277)
    I can not imagine how floppies have survived there, why the heck they were even chosen in a year they were on a certain dying slope. But wifi for critical infrastructure? Really? Wifi can so easily be interfered with and we put trains with people dependent on such?
  • Wire loops? (Score:4, Informative)

    by spyfrog ( 552673 ) on Wednesday October 23, 2024 @12:24AM (#64886361) Homepage
    What do they mean with wire loops here? Is it the action where the physical train short circuit an electric loop in the rails to indicate that block as blocked? Those system are the most secure and most reliable you can have - the alternatives are much worse. We have several kinds of position system on the railways here and the other kind of systems are much less reliable
  • by DrXym ( 126579 ) on Wednesday October 23, 2024 @03:24AM (#64886535)

    I realise this story conflates two things and is referring to a full system upgrade. But if they wanted to get off floppy disks, it would be trivial to code up some arduino devices for that purpose. There are numerous such projects for retro gaming that could repurposed to remove the magnetic media entirely.

    • This has been discussed at length before. If the system is only using the floppy disks for boot, what you suggest might be viable. For safety critical systems, though, emulators that work great for gaming aren't necessarily viable. Slight timing changes, which are always present in such emulators, might mean the difference between correct and incorrect functionality. Even gaming recognizes this as many records won't be recorded if played using emulation and not the original hardware.
      • by DrXym ( 126579 )

        If they were safety critical they wouldn't be using floppy drives at all, at least during operation since they would be sensitive to failure at any time from mechanical failure, power spikes, vibrations and other such things. Safety is very much concerned with reliability or failing to safe, so if an entire train was relying on a spinning disk at every moment it would be neither. So it is booting up from them, and operating from memory thereafter.

        Also, I wouldn't be suggesting just throwing an emulator boar

  • 30 years ago, I worked on a system already a quarter century old, that, among other things, sent data intercontinental via fiber optic cable.

    They had rented an 8 bit wide pipe through the cable at whatever speed, so I had fun with reading and writing a COM port across the Atlantic. Or was it 9 bits? I dunno.

    Time moves along. I thought it odd it spoke of being slower than a wireless modem. Uneless they meant such back in the day.

  • No, they are not spending $200M to upgrade from floppies. They are revamping the whole system. Incidentally, the current old one uses floppies.

  • Model Railroaders have been perfecting computer control and Dispatching programs for as long as there have been computers. I'm sure there are several open-source solutions available. The Certification of the software is expensive. That's what the $212-million is for, paperwork.
  • In 20 years from now they will fail to update it.

    40 years from now we will see the same story of an archaic train system stuck in the past using clunky tech called wifi.

  • That's a huge amount of money... for what amounts to a whole bunch of goteks !

  • "San Francisco announced the rail system upgrade planned 10 years ago will now end up costing 500 Million dollars"

To communicate is the beginning of understanding. -- AT&T

Working...