Slashdot Log In
Low-Bandwidth, Truly Remote Management?
Posted by
kdawson
on Tue Nov 04, 2008 03:58 PM
from the going-old-school dept.
from the going-old-school dept.
kaiser423 writes "I'm looking to integrate some highly critical solutions into what would essentially be a remote, moving datacenter. No operators will be allowed at the site, and we may be able to have a high-speed INMARSAT data link. As a backup, we're planning to have multiple redundant low-speed Iridium data links. Essentially, we're looking to be able to power up/down and reboot some computers, and be able to start/stop some programs. We're willing to write the terminal interfaces necessary for our programs, and possibly do the remote desktop thing with some of our 3rd-party programs. But what is out there that would give us this type of access, work robustly over a high-latency, low-bandwidth stream, and would be tolerant to intermittent network outages? Please hold the pick 2 of the 3 jokes, I know they're contradictory goals; I'm looking for a compromise here! These boxes would regrettably nearly all be running Windows (with some VxWorks). Does anyone out there remember those days, and have any solutions that they preferred?" Read on for a few more details of this reader's requirements.
We've been looking at remote in-band and out-of-band management solutions, and really have found a ton of products. However, the "low-bandwidth" solutions still exceed our potential Iridium bandwidth (~10Kbps). Even if we have the INMARSAT link (192Kbps sustained, higher burst), a number of these solutions would hit that limit. We're starting to look at going old-school with some terminal-style applications, but haven't found much of a market for it; it seems to be a market that died with 56k modems. PC Weasel looks kind of like it might work, but the demo doesn't work for Windows.
Related Stories
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
IPMI + DRAC or similar? (Score:4, Insightful)
you can do an aweful lot with IPMI, i.e. power cycle, and a remote access card that supports ssh can do the rest, alternatively a TTY terminal and pipe your consoles to serial ports
oh, windows? nevermind
The ONLY Correct Answer (Score:5, Insightful)
It's no joke. Embedded remote access cards like iLO from HP or DRAC from Dell are the only real solutions.
It would be painful to use their console viewing features over a low speed satellite link but, it would be no problem to power cycle, collect statistics and more. You can even forgo the web interfaces and use ssh on many of them.
Parent
Re:The ONLY Correct Answer (Score:4, Informative)
Agreed.
I remotely manage 50+ ProLiant servers this way, mostly via SSH to the iLO. Unfortunately since you want/need to run Windows on them you'll have to go graphic mode (via web interface and a Java app) to the OS. (I manage Linux servers so I can do it all via CLI.) You can even do remote installs via virtual media that mounts your local CD/DVD drive (or ISO image) on the remote, although that'd be painful at your speeds.
Parent
Re: (Score:3, Interesting)
Isn't MS Server 2008 heavily scripted? My understanding is that you now have almost as much control in a 2008 environment with no GUI (CLI only) installed as you do with a GUI.
http://www.microsoft.com/windowsserver2008/en/us/powershell-faq.aspx [microsoft.com]
This may be the way to go.
Re: (Score:3, Informative)
Powershell can be installed on Core, it just isn't supported. Also 2008 R2 will have powershell.
That being said, installing in full mode, you can use powershell in a supported config and manage it using that. It is awesome.
Powershell (Which you can install on 2003 as well) + HP ILO and you should be set.
I would like to say stay away from Dell's DRAC if possible. I've worked with DRAC III, IV, and V and they all *suck* compared to ILO 1/RILOE/RILOE II/ILO2
Re:The ONLY Correct Answer (Score:4, Informative)
Painfully untrue, especially in Server 2008 (for which the core install doesn't even have a GUI). There are scripts, tools, and other things that make remote administration of windows possible in many ways that were much harder, previously. No GUI needed.
Parent
Re:Windows Command Line Interface (Score:2)
RDP (Score:5, Informative)
There's a surprising amount you can do from the command line within Windows these days. For UI access, RDP beats the common alternatives hands down, even if you log in just to use a command prompt remotely and thus have console state stored between sessions if the connection goes down. Have you actually tried this?
I wonder if anybody can put some numbers on the latency and bandwidth? I spent four months in China maintaining Windows servers in California via RDP. With latency often around 600-750ms and packet loss, it was painful but still usable. I was even contending with nested RDP sessions (RDP over the VPN to a machine in an office in CA, and then RDPed from there to a colocation facility).
DTMF (Score:5, Informative)
"Essentially, we're looking to be able to power up/down and reboot some computers, and be able to start/stop some programs."
Dial in using the telephone system and use a sequence of DTMF tones on your telephone keypad to carry out a task. This will be low bandwidth (about 2,700 Hz) and low cost.
Ganty
Sat DTMF (Score:2)
Right idea but if he's using sat links? Then that tells me that phone lines may not be an option. Now shortwave or some other form of wireless however...
Now for the OP I'd recommend asking himself what exactly needs to go over the link and what can stay local? Power up/down and reboot can stay local. Service start/stop? That depends.
Re: (Score:2)
His low-bandwidth link is Iridium - i.e., satphone. I'm sure with one of those things a regular voice call works. BTW, last time I checked, the datathroughput of Iridium was aro
Re: (Score:2)
It's an option, yes, but likely not a viable one unless the admins are looking at the very simplest of tasks, e.g. warm-booting servers.
Here's a question that might prove relevant. How does NASA handle very similar requirements for their interplanetary missions? This might be a good time to open some dialog with the good people at Goddard or JPL and see what sort of ideas they might have.
RS-232 Serial Port (Score:5, Informative)
Not many options (Score:5, Informative)
Tivoli Management Framework had configurations designed to work with satellite links as slow as 16k. That solution was for monitoring and configuration management though -- not what you want.
Your big problem here is your expectations. Remote Desktop over a slow-speed, high latency link just isn't viable. Anyone paying the megabucks required to support a field-deployed solution will not be happy with the crappy service you'll ultimately provide.
You need to extensively model how your application works and develop appropriate procedures, runbooks for your remote operators and a toolset of programs or script to provide support for this "critical" solution.
Two simple words (Score:5, Informative)
Two words will get you far in this situation, Command Line. Low bandwidth, latency tolerant, and generally asynchronous. If you can get any tools with a command line option, embrace them.
GUIs suck, and they suck more over the conditions you describe. Avoid them like the plague. Also, think about mirroring the files you need to manage and editing them locally, then uploading them when you are done. Not always possible, but if it is doable, it can make your life a lot easier.
Scripting is your friend here.
-Charlie
One simple acronym: "ssh" (Score:2)
Came here to post this.
The article has by now been tagged "ssh", which should be the obvious answer indeed. Even in Windows server editions this is a perfectly fine option these days - you should be able to do mostly anything that mat
Maybe we need more details? (Score:3, Informative)
http://www.avocent.com/products/serial-consolemanagement.aspx [avocent.com]
http://www.raritan.com/products/serial-console-switches/ [raritan.com]
Plug one of these in, then connect a serial cable to your servers. Many include a modem if you have a pair of copper wires for a phone line so you can keep it out of band.
Re: (Score:2)
Some possibilities... (Score:3, Interesting)
a: Remote management cards often have command line interfaces for resetting, system health, etc, through SSH. True, SSH with 800ms RTT times is a pain-in-the-ass, but if scripted, should work fine.
b: Once you can power cycle/machine health remotely, now you use SSH to connect to a command line shell on the system itself (yes, even windows) and do all further tasks from the command line.
Control the power. (Score:2, Insightful)
I think it is important that you have all the equipment you can on IP addressable Ethernet Power strips so you can physically cycle the power remotely independent of higher level computer control. Something like this: Power Strip [leunig.de]
There is no substitute for the ability to toggle the most significant bit--for sure.
It sound like this is for Science in Antarctica.
SSH or stunnel? (Score:3, Informative)
http://www.bitvise.com/winsshd [bitvise.com] It does the job connecting all kinds of platforms/client implementations. It does PKI too.
HP's Compaq line of servers has **excellent** remote admin capabilities.
Push the whole thing over an stunnel and you are good to go.
Implementation is another issue. Publish an email if your budget supports consultants. Errmm. Well, it looks like slashdot is taking the place of a qualified expert, so good luck with that.
A couple of things... (Score:5, Interesting)
You may be able to run RDP across even the low speed links - choose the bare minimum screen resolution and color depth possible. It will still be pretty frustrating and slow, but you could use it with enough patience. Or you can run VNC, though I believe its performance will be less than RDP.
For CLI access, install an SSH server on the Windows box. If your code runs as a service, you can interface to it through a CLI client. It's some development work, but possible.
For power, I like APC's smart power strips. They support HTTP and SSH access.
Whatever solution I used, it would have to be run over an encrypted satellite link.
Re: (Score:3, Informative)
Using TightVNC (high compression) and the DFMirage driver helps a lot, as does forcing your VNC viewer into 256-color mode (something I also do for RDP). I don't know about constrained network bandwidth, but on a LAN these things make VNC just as fast as RDP IMHO.
Tip for using 800x600 -- if you set the Taskbar to auto-hide, you will still have just enough room to click OK/Cancel on tall dialogs.
Back to the submitter -- seriously, Tel
VNC/RDC/NetOP (Score:5, Informative)
As for the iridum setup, you might want to check those speeds again. When we looked into it, they were only able to offer a 2400 baud serial connection over the Iridium system. That is REALLY slow, and with high latency. We decided against going with the backup.
I'm not sure where you are planning on deploying these setups, but maybe a cellular modem would fit your needs more. They are fast, low latency, and comparable in price. The only hitch with them is that there is no SLA; just best effort. If you are out of coverage area, try to hook up a high gain antenna to the cell modem and try again.
I would just use remote desktop tools to manage your servers. VNC especially works quite well with low bandwidth, high latency connections.
One thing to let you know though. Make sure that you have someone that can go to the site and has access to the network equipment. We have almost 20 sites like this, and about 2 go down a month. The sat-modems or sat-controllers sometimes need to be rebooted, and having someone near the site to do that can save you guys A LOT of money. When we have to send out a tech, it costs around $2,000.00 due to driving charges.
go for redundancy (Score:2)
IP aware KVM (Score:2, Interesting)
Get an IP aware KVM.
Using these you are separated from the operating system completely. You can see the machine boot, get into its BIOS, do a power reset (with a compatible power strip).
They have encryption and use a Web interface. Some have a fat client.
And try to run things from a command line as much as possible. Have the machine start a full screen command session upon boot, and hide the task bar. That should minimize the initial screen scrape.
Its the next best thing to being there...
Re: (Score:2)
"Its the next best thing to being there..."
except for when you are on a low-speed link in which case it is the worst possible solution.
Re: (Score:2)
I've connected to mine over a 9600 cellphone DUN link and while it was pretty slow, you could get things done. I wouldn't want it as the ONLY option, but it's worth looking into as a backup.
Re: (Score:2)
+1 on the IP KVM. The one I have uses VNC, and if you have USB cables hooked up & your BIOS supports it, you can actually boot from remote media (ie, point your client to your CD drive, and the KVM emulates the bootable media on the other end). That would be painful over a sat link, but it would be nice to know it was an option if needed. The nice thing is it doesn't mean that you can't have other ways to connect to the individual boxes, it's just another option, which is always good.
Peppercon eRIC boards (Score:2, Informative)
We've been extremely happy with the eRIC [daxten.com] remote management board from Peppercon for all our lights-out remote-datacenter Windows machines. We use it over landline dialup modem (33.6) but it will work fine over any serial link you can throw at it, as long as you're patient. Full graphical remote console, remote IPMI, remote reboot, remote poweron, indepdent power supply (optional), it's great. They're a little hard to track down in the US, but I believe Raritan distributes them now. They're not cheap, bu
Your Sinister Plan, Sir? (Score:5, Funny)
Without knowing what you're trying to accomplish, it's hard to give any real advice. Automated spy van? Unmanned laser plane? Knight Rider style service truck? Continually running train with nuke launcher? There are only so many things you can do with an intelligent roving unmanned platform.
Re:Your Sinister Plan, Sir? (Score:5, Informative)
Parent
RealWeasel? (Score:4, Informative)
Re: (Score:3, Interesting)
We call these frame buffer cards.
I remember when the PC weasel first hit and I was like this is so awesome.
Now, there are cheaper bolt on or integrated solutions.
Built into Windows? (Score:2)
The advantages of using RDP are several: it's included with Windows so there are no additional costs (meaning no money is wasted on licenses if you decide it sucks and w
Why? (Score:2)
It's really difficult to get a feel for this question without knowing why anyone would want a 'remote moving datacentre' that isn't well connected to anything else. Why can't the computers be sited somewhere less troublesome? Would virtualising the machines to get round the booting/power on/off issues help, or is there some reason that this can;t be done?
Re: (Score:2)
SNMP (Score:2)
NetSNMP works even on windows. It is readily extended using shell scripts (to support your frequent reboots).
Of course, SNMP also provides remote monitoring as well.
I hope you have the ability remotely power cycle unresponsive boxes.
Virtual Machines (Score:2)
You could run Windows as a virtual machine on a Linux box and use the command line tools to manage it.
Start with the hardware... (Score:3, Informative)
Start with your server hardware. Most Dell servers have a Dell Remote Access Card which allows you to get a full console (including BIOS and power on options) via web page. Performance of the full GUI over a slow link is marginal however. I'm sure other server vendors have similar options.
Also or alternatively, look for BMC controllers (http://en.wikipedia.org/wiki/Baseboard_management_controller) and IPMI interfaces (http://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface).
From there address further needs with RS232, Telnet, SSH, etc. Step up to RDP and VNC for GUI needs.
RDP Will work (Score:2)
Those of us using RDP (and ICA - but a Citrix installation just to get the ultra-low bandwidth for ICA might be overkill) back in 1998/1999 can tell you that it can work over a 9600 bps connection. Just set it for 16 color. You can also (as other posters have mentioned) do PowerShell for a lot of things. You can stop and start services and restart a Windows box from the command line (net stop [service]/net start [service] and shutdown /r respectively) and depending on the data may be able to transfer i
HTTP? (Score:2)
I will probably get yelled at, but the simple answer to me seems to be writing a web app. It was stated that they are willing to write terminal applications, so there is obviously some programming ability present.
So why not write web apps. If you write your HTML well, then you can run it over pretty low bandwidth and all the work can be done by the server. There are still sites that work with 56k modems so it is possible to create low bandwidth sites (although most have stopped bothering).
There isn't a s
IPMI and RDC (Score:2)
Although many here are anit-Microsoft folks, I've got to admit Microsoft's Remote Desktop Protocol is *very* robust over high-latency, low-bandwidth networks. We've had many employees use it over satellite (DirecPC, usually) without any issue at all, when ssh or OpenVPN would fail.
As far as remote reboot, etc, IPMI is your friend. Most modern server hardware includes at least a rudimentary support for it on the baseboard, with more capable optional cards available. One example is that Dell servers have a
VxWorks (Score:2)
I don't know about Windows machines, but I heard that certain robotic probes [cmu.edu] run VxWorks and are remotely controlled via a low-bandwidth, high-latency connection. Those devices have a lot of programmed autonomy and fail-safe built in. And they don't run Windows.
Be truly innovative (Score:3, Funny)
This may take a while as threading the mail system tends to cause kinks that have to be sorted autonomously by remote postman protocol.
When it arrives, plug in and use normal LAN remote control technology. Thanks to XKCD [xkcd.com] for the idea
Something I have expertise on.. (Score:3, Insightful)
I've seen people watch a slingbox stream over these things with no skipping, in a dense area. In a sparse area like what you've described, no one else will be on your spot beam, so the entire bandwidth of the beam is basically yours to use. It's really not an issue.
Also, if this doesn't work for some reason, easy solution for the CLI that no one wants:
1. Write CLI
2. Write Client-side GUI for CLI so that customers think it's new technology
3. Profit!
Did the same thing... (Score:3, Informative)
...only I used BGAN instead of standard INMARSAT (which may actually be what you're referring to here). In instances where I needed GUI access on Windows boxes I found a very workable solution: Installed 1 Linux box with FreeNX server and put RDP client on that box. I'd NX into the Linux box and then RDP into each Windows box from there. Absolutely workable over even a crappy connection.
Re: (Score:2, Insightful)
I wouldn't fight the jokers defending ridiculous specs like low-latency low-bandwidth remote windows implementations. Use telnet, only call it secur-link 2008 in the specs. The joke is right back at them.
There are plenty of commercial ssh implementations for windows.
Re: (Score:3, Interesting)
Re: (Score:2)
Sorry, but "highly critical solutions" and "Windows" does not really go together.
You don't seem to be heard of "Project Managers" in all their variety...