AWS Customers Rack Up Hefty Bills For Moving Data (theinformation.com) 64
An anonymous reader quotes a report from The Information: There are a lot of ways companies can rack up high bills for using cloud services, sometimes unexpectedly. One particularly stiff expense is the cost of shifting data from one cloud provider's servers to another provider, or to a company's own data center. The Information has learned just how much some companies have had to pay for these "data transfer" costs, as they're called. The chart above shows how much 10 of the top customers of Amazon Web Services -- the dominant cloud provider -- paid for data transfer services in 2017 and 2018. The chart, which is based on internal AWS sales figures obtained by The Information, show that data transfer charges for one customer, Apple, approached $50 million in 2017. That represented about 6.5% of Apple's total AWS bill of $775 million for that year, the sales figures show. Seven of the 10 companies saw increases of at least 50% in their AWS data transfer bills last year compared to the year before. The reason for the high bills could "stem from growth in the number of users on a company's web service, longer-than-average usage sessions and the addition of data-intensive features such as video," the report says.
The reason for the high bills (Score:4, Insightful)
is vendor lock-in.
Re: (Score:3)
Re: (Score:2)
Re:The reason for the high bills (Score:5, Insightful)
is vendor lock-in.
Nope. Vendor "Lock-in" would mean you CANNOT move to another provider, which is obviously not the case here. "Vendor lock-in" is like when Apple sells you something, and when you want to move to Android, you have to let go what you bought. A lightning equipped docking station for example.
In this case, the price may seem pretty high, but knowing that Apple certainly read the ToS (and for fuck's sake, the price list!), they knew they would have a price to pay for data transfer.
$50M of data transfer is a lot, but not that much, considering that if Apple hosted all this on-premise, they would have needed a frickin big Internet pipe to serve from there. Of course, when you outsource, the provider makes a profit. Amazon is not a charity.
At 0.09$/Gb, this means they would have transferred approx 530PB of data. This is a whole fucking lot of data. Almost 1.5PB/day 17Gbps at a constant rate. Find a provider that will offer you this kind of Internet pipe for less.
Re: (Score:2)
Oops, I forgot to multiply the last number by 8. Its 140Gbps. Damn, that's pretty big, and we're not talking about peak rates here!
Re: (Score:3)
You don't believe in the concept of economic lock-in, where a provider charges discouragement rates to keep people from leaving their service?
$50M in data transfer charges seems crazy. I'm thinking you could custom-build a trailer with power, cooling and 50 PB of storage for a couple of million dollars and drive it back and forth to a data center for under $10M.
Re: (Score:3)
You don't believe in the concept of economic lock-in, where a provider charges discouragement rates to keep people from leaving their service?
Discouragement rate for leaving their service would be, for example, charging double the data transfer rate for taking a copy of a VM outside of your account. Even then, moving 1TB out of your account would cost 180$ instead of 90$. Not exactly something I call "lock-in". And I have not heard anything about a higher rate for moving your data out over serving it to outside users.
Yes it would be frustrating, and you could moan all you want about it. But it would certainly be somewhere in the ToS, so it would
Re: (Score:2)
Re: (Score:2)
You know I meant Amazon. Not Oracle. I have a problem with my CPU this morning when I comment here on slashdot.. I forget words, forget calculations, crop my comments... Damn, what's right with me?
Have you considered moving your brain to AWS?
Re: (Score:2)
Hence the old saying about a station wagon full of mag tapes.
Of course that would presume that you could find which cloud data center had your data, and be allowed to go inside and hook up to it. "Sure, we just want to copy off our own data, not going to mess with any of the hundreds of other servers in there, scout's honor!" or "Do you mind if we plug this fiber cable from our data truck into your backbone for a while?"
Re: (Score:2)
I'm thinking if I have that much data at their data center they will go along with this given how much I spend in their data center.
Re: (Score:2)
Re: The reason for the high bills (Score:2)
"$50M in data transfer charges seems crazy. I'm thinking you could custom-build a trailer with power, cooling and 50 PB of storage for a couple of million dollars and drive it back and forth to a data center for under $10M."
And AWS has such a product/service.
But, that wouldn't have worked here, because the data transfer was most likely mostly "serving massive amounts of ever-changing data to Apple's users"
There are real costs to serving > 100Gbps of content to users, on average, with great reliability.
Re: (Score:2)
. I'm thinking you could custom-build a trailer with power, cooling and 50 PB of storage for a couple of million dollars and drive it back and forth to a data center for under $10M.
I know, it's almost as though latency was a factor too.
Re: (Score:2)
Nope. Vendor "Lock-in" would mean you CANNOT move to another provider,
There is never a never. Vendor lock-in means it is very expensive to change, not that it is impossible. If you buy a Motorola radio system, for example, you are locked in to Motorola -- until you spend the money to replace it.
If you put all your data into the AWS bucket, you are locked into AWS -- until you spend the money to get it all back out of AWS.
I was asked to investigate the costs of moving our data into AWS. The storage was really cheap. The open-ended costs were accessing the data outside of A
The reason for the high bills... (Score:3)
Re: (Score:3)
Banks/Credit Unions also do this by charging you a fee to transfer funds to other financial institutions
Re: (Score:2)
I haven't run into a credit union doing this, they want your business. Banks however? Yeah...banks everywhere do this.
Re: (Score:2)
My credit union does, and the fees are not cheap.
Re: (Score:2)
Willing to post what CU this is? I'd like to avoid them in the future especially if it's in the US South or in Florida. Unless FirstOntario(Canada) has changed in the last couple of years, they don't do that. Far as I know too, they also offer free switching of your account's e-bills to the new bank.
Re: (Score:2)
Wire transfers have a hefty fee, but there are other ways to transfer funds between financial institutions. It is really no different from paying a bill from your checking account online. I use multiple financial institutions and none of them charge a fee to move money between them. It usually takes a day or two to clear, maybe a wire transfer is instantaneous.
Re: (Score:2)
Big companies generally don't go full hog to switch from one cloud to another unless they have a political reason for doing so. Instead what is probably happening is making redundant copies of their data across multiple clouds (and within the individual clouds as well). Why make a single cloud provider a point of failure when you can have backups across multiple clouds.
---
Artemus Harper
Azure Engineer
(This opinion is my personal opinion and not reflective of Azure or Microsoft)
Re: (Score:2)
It is probably the result of an intentional decision by the cloud provider to make it more difficult to move away to another provider.
Bandwidth "up" is simply what costs. For the cloud, that's the bandwidth for data flowing out of the cloud. That's not any more of a scam than any internet service anywhere.
My story (Score:5, Informative)
I am not a large customer by any means, just a regular person looking for cloud backup of my data which measures about 700 GB at the moment.
The first service provider I went to, a couple years ago, was Amazon, and they estimated around 10 bucks per month, then within a few days since I set up the service and started using it my monthly estimated bill jumped by a couple dollars per day; after 5 days it was a bit over 18 dollars, at which point I canceled the service, paid them the owed sum and never touched Amazon's cloud again.
Currently using Backblaze B2 which costs me 3.5 dollars a month for the same data.
Tarsnap (Score:4, Informative)
Currently using Backblaze B2 which costs me 3.5 dollars a month for the same data.
Tarsnap tends to be cheap, too.
It still relies on servers hosted by Amazon (as opposed to back blaze who build their own storage pods. And even share tests results - kudos to them), but the dedup algorithm is good (you only send your whole 700GB once, the first time. Then you only send the changes), so you won't have high bandwidth costs.
Bonus point: the client of tarsnap is open source, and has decent cryptography written by Colin Perceival (of "Scrypt"-fame).
Re: (Score:3)
We're not a large operation by any means but we're significantly larger than what you quote. We moved our on-prem infrastructure first to a managed data center and then on to cloud (split between AWS and GCP) because of the ability to pay for what we're using and dynamically scale.
Moving from on-prem to managed data center saved us around 50% a year on infrastructure and infrastructure management costs with roughly the same ability to deliver.
Moving from the managed data center to AWS/GCP dropped our costs
Re: (Score:2)
You're talking solely about backups and storage here; that's literally only a small part of our business case and your statement makes an incredibly extreme viewpoint on that small part about the value of cloud which makes its value pretty narrow.
Re:My story (Score:4, Informative)
Not sure what service on Amazon you are using but Glacier is cheap. It should be around $2.80 per month for storage. The problem is the retrieval costs are higher.
Re: (Score:2)
Heck, you can't reasonably restore from Glacier if you lose everything - you're only allowed to download some small percentage of your data. You restore from Glacier by promoting to regular S3 for the download. At which point, you're really going to notice the price per GB for the transfer if you have several TB. But it's orders of magnitude cheaper than a data recovery service pulling data off a failed HDD.
Re: My story (Score:2)
If you have to pay so much to retrieve your backups, how do you verify them? Backups are useless unless you can verify them regularly. From the sound of it you'd have to pay a fortune to do this long term with cloud providers.
Re: (Score:2)
Once you know it's store properly, it's not going to go bad. For most purposes, you can verify them by random sampling.
Re: (Score:2)
Re: (Score:2)
I've been happy with Wasabi. $6/terabyte/month, no entry/exit fees, but any data stored is billed for three months, so if someone throws two terabytes on and deletes them, they will pay the $36. But compared to ingress/fees of other providers, this is a reasonable requirement.
Oh, it also uses the AWS S3 protocol.
Re: (Score:2)
I'm on Backblaze for the same reason: Transparent pricing scheme. I'm happy to pay them for half a dozen machines, because they do they job I pay them for for the price that we agreed upon. No hidden fees, no tricks.
They did the math (Score:2)
Make the cloud just barely cheaper than buying your own hard drives and storing it locally.
Re: (Score:2)
Yeah, my NAS backs up to my other NAS which is in a different location.
It's trivial to set up and highly cost effective.
Well, Duh (Score:2)
What does this include? (Score:4, Informative)
Re: (Score:2)
Within the same AZ, it's free as long as you're using a private IP address. If you're using a public IP or elastic IP, you're going to get charged.
Moving data across AZs is charged at a low rate (0.01/GB) but it still costs money.
Re: (Score:1)
Much ado about (almost) nothing (Score:3)
6.5% of Apple's total AWS bill
If my enterprise's IT bill could be broken down in any way such that the cost of data transfer was only 6.5%, my management would be ecstatic.
Moving data is expensive, and what's described (in TFS, at least... I'm not inclined to actually spend my time reading TFA) is the most difficult part of moving data - moving it to lots of customers. As I understand, Amazon has several options for moving large amounts of data in and out of their cloud, essentially involving shipping hard drives between data centers, and billed at a much lower rate.
Re: (Score:2)
Re: (Score:1)
Moving data is expensive
No, it's really not, especially not bulk transfers. 10Gbps global transit, i.e. moving data between anywhere in the world and a major data center costs less than $900/month. That's what Hurricane Electric will admit publicly, but transit prices can be haggled. At 30% average utilization (although you can use 100% if you want), 10Gbps is roughly 1 PB (1000 TB) per month in each direction. That's $1/TB.
Re: Much ado about (almost) nothing (Score:2)
"If my enterprise's IT bill could be broken down in any way such that the cost of data transfer was only 6.5%, my management would be ecstatic."
I hope you meant storage, not transfer? Are you upgrading switches every year or something? Tons of private circuits? Your entire IT budget is 10x your Internet service?
Or do you mean it's like 9% and you'd be ecstatic with 6.5% because you have some weird accounting genes? Most regular places are grappling with yuge software licensing and storage hardware costs
Re: (Score:3)
Exactly. I get bored fielding questions or comments from people where they opine about transfer costs of data in/out of AWS, particularly research data which is practically free if going over I2 and you're not pathologically moving bits back and forth.
Compute services and some complex network configurations dominate. If you really want to complain about AWS then bitch about how they don't provide simple limiting tools to avoid running up exorbitant costs for stale resources. They do provide tools but it
Cloud provider charges money for services... (Score:2)
News Flash: Cloud providers charge money for services they provide... It's not like these are hidden charges -- they are pretty clear that there is a cost associated with it, and it's pretty easy to calculate.
If you are transferring data within the AWS network, there shouldn't be a charge. It's only when you leave the network. Just make sure you are using the internal IP addresses when you do so, or it looks like egress traffic.
Also: Water is wet, sky is blue. (Score:2)
The most expensive thing in a remote computing service is outbound transit? Shocking!
Re: (Score:1)
The IT landscape transisions again and again. (Score:2)
1980's - Mid 1990's Timed Computing. Where companies who cannot afford large mainframes would rent time to use them.
Mid - 1990's - Mid 2000's Companies had data on their PC's or with dedicated servers. As PC's and Servers have became powerful enough and affordable enough to meet their computing needs.
Mid 2000's to current. Cloud service technology in general is more efficient at system utilization, and having your own servers meant a lot of maintenance on them from IT staff, who could be focused on more
Re: (Score:2)
Cloud service technology in general is more efficient at system utilization,
For the same reason that makes buying a plane ticket cheaper than owning your own jet - with the same shortcomings and exceptions. If you fly a lot, and if you sometimes really, really need to fly right now, from a specific place to a specific place and one of them isn't a major city - then that's no longer true.
In fact, after spotting seven people from the same company on the same flight I once made a few calls and did the math and went to HR to inform them that if that happens again, chartering a learjet
Re: (Score:2)
The real fun is when you are working in the gray zone. Where it is only slightly cheaper to charter a flight, or go with your own servers, but there are hidden costs that you are afraid that you didn't factor in. Such as tipping the service staff on the learjet (I really don't know if that is a thing or not), or paying extra for inflight meals. Or in terms of hosing in your company. Having to upgrade your AC system to deal with server heat.
Convince is a factor too. Booking 30 tickets on one plane, vs tryi
Re: (Score:2)
We have a word for that and it's business risk. We know how to factor insecurities in, it's not a new science (just new for IT for some reason, I've been trying for years to get people out of the stone age on that one).
But yes, the grey zone is the interesting one. When people need to make decisions under uncertainty. When all the psychological effects come into play. The endowment effect. Prospect theory. Risk aversion...
Regions (Score:2)
It is not just between providers either. You can rack up massive bills by moving data between regions of the same provider - about as much as switching provider.
surprise, surprise... (Score:3)
Paying a company that also wants to make a profit to run servers for you can be more expensive than running your own servers.
No who could have possibly predicted that ?
Of course they scam you on extra charges. In a world where we more and more go by price and price alone and everything needs to be cheap, cheap, cheap, those additional "service fees" or "upgrades" or DLCs or micro-transactions or whatever are the only way a company can beat the competition on price and still be profitable.
And yes, you didn't read the fine print. Poor you.
In our experience.. (Score:1)
This isn't the first time Amazon has imposed rate hikes from moving data to other cloud service providers. On paper it's typically considered a premium fee to cross transfer from AWS to competing cloud providers.
One key recommendation when moving away from AWS is if data is also stored on-prem then quietly shifting data from on-prem to new cloud service is highly recommended and far cheaper than the alternative.
Idiot cloud blinded bosses (Score:1)
You only find these when you start digging through how some of the services are charged, they give you the service dirt-cheap to get the data in and build it up but they charge a ton for shifting it out. The thing is that they don't hide any of this, all the prices are available you just have to spend the time digging through the reams of price info to find them. To many PHBs have dictated that AWS must be used 'cos "the cloud" was the latest fad about 2 years ago, they demanded all projects be put into AWS
So the news is? (Score:2)
Companies move more data, pay higher bills? More news at 11?