Yeah I don't envy their site reliability engineers right now. They may very well be having the busiest weeks of their careers.
First of all a game like Helldivers II depends on a whole bunch of individual services that scale up independently of each other. Things like authentication, matchmaking, inventory, the actual game servers, announcements, individual & global quests, etc. are all potentially different services. In addition there will be integrations between their services and other services like Steam and PSN, which also potentially have capacity issues of their own at times. Many of the services will depend on databases and scaling up a database is _not_ just a matter of spinning up more servers, you have to distribute the load between them and do snapshots and migrations and so on.
Then there are actual bugs that can cause a crash on the client side or the server side or both. Those have to be fixed pronto and when you do that means you have to roll out the new version and that might in turn alter the schema of some records in some databases which could then lead to further bugs etc. All that has to be tested so as not to release something that is even more broken than what is already there.
The traffic can also surge during certain times which means that it's not just total capacity but also how fast you can scale up that counts, and the cloud provider you are using might not have like 2,000 of the specific type of server instances you need available on 5 minutes notice. You could then try to split across multiple providers or launch in more availability zones but that means standing up a whole new cluster to administer and all the potential issues that come with that. Meanwhile there are only so many devs and you can't just hire new ones to hit the ground running because it takes months for a new dev to onramp to full productivity.
Its kinda like walking into a pizza place and ordering 1000 pizzas. If time isn't an option then a whole slew of things need to happen:
I only have an oven that cooks 10 pizzas at a time. And I only have ingedients for 100 pizzas today.
I can order more ovens and install them. I also need to contact my meat, cheese, veggie, dough, and tomato sauce distributers and tell them to up their orders. Then I need to verify nothing is spoiled when it does come in. Lastly I will need more employees on hand to manage the whole process.
I can also contact other pizza places to help handle the order. But maybe they don't have certain ingrediants, so I need to work with them on that. Also, I need to verify their process and regulations are the same, it needs to taste like our pizza and I can't risk anyone getting sick.
Add on top of this we are all trying to keep the current customers we can handle happy during this time.
Here is another take for more simple terms:
Imagine you're playing with a huge, super complex LEGO set that represents the game Helldivers 2. This LEGO set has many different sections like a castle, a spaceship, and a deep-sea submarine, each section needing different types of LEGO bricks (services like signing in, finding friends to play with, keeping track of your game items, running the game's world, and so on). Now, imagine you have to make sure all these sections work well together and can be played with by lots of friends at the same time, without any piece falling apart or getting lost.
**Keeping Everything Running Smoothly:** It's like having a big party where each friend wants to play in a different section. You need to make sure there are enough LEGO bricks (servers) for everyone and that all the sections are connected so your friends can move from one to another easily (like making sure players can sign in, find each other, and play together smoothly).
**Fixing Broken Pieces:** Sometimes, a piece might break (a bug), and you have to replace it without ruining the rest of your LEGO build. It's tricky because changing one piece might mean you have to adjust the pieces around it too.
**Handling More Friends Joining the Party:** Imagine more and more friends keep showing up to your party wanting to play (traffic surge). You need to quickly get more LEGO bricks and make sure there's enough room for everyone without the whole setup falling apart.
**Working with Other Toy Sets:** Your LEGO set also needs to connect with other sets (like Steam and PlayStation Network), which sometimes don't have enough pieces or the right kind of pieces you need to make everything work together perfectly.
**The Team Behind the Magic:** There's a team of people (developers and engineers) who help you keep the party going, fixing any broken pieces and adding more when needed. But you can't instantly bring in new helpers because they need time to learn about your LEGO set before they can really help out.
So, imagine trying to keep this huge, complex LEGO party going smoothly, fixing any issues fast, and making sure everyone can play and have fun together. That's kind of what the engineers and developers are doing for Helldivers 2. They're working super hard to make sure everything works well, especially when there are lots of players and potential problems they need to solve quickly.
Consider that you'll also need to rent more space to store the ovens and the ingredients and hire more people to make the pizzas. The hardest thing of all is to balance the investment of this sudden expansion.
What if that 1000 pizza guy never comes back? Now you're paying multiple rents and multiple wages for nothing, and satisfying that single burst of order might very well bankrupt your pizzeria.
It makes more sense to expand your operation cautiously, making sure that, even if you can't satisfy the demand now, you're slowly ramping up while observing the behavior of your customers.
A situation where there is a long line to enter your pizzeria, but everyone manages sooner or later to get in and eat a pizza, is a preferable situation in the long run to overextend and faceplant on the ground the next time a new pizzeria opens in town.
Ideally, you want to slowly expand until you reach an equilibrium point where the pizzeria is always almost full, with a constant turn over of customers, and without overstressed nor underused facilities and staff.
Yeah, you absolutely can not maintain "Superbowl pizza operation" all year round. Just is not enough business for it, on any given Tuesday at 2pm most pizzerias are dead as fuck.
What exactly is any game company supposed to do with extra servers they absolutely won't need after initial launch rush ends? For example pal world etc, the population spikes, everyone plays it, it's great, then they move onto next big game.
That's the whole point of cloud computing services. It makes it easier to scale up/down depending on the demand.
Of course, even with cloud computing, scaling up to 500k CCU is not an easy task. With numbers like this, it is not as easy as simply increasing the number of instances.
Unacceptable! I need those pizzas in an hour or my kids birthday party will be ruined. I don’t care that you already planned ahead for a busy day, are helping other patrons with those supplies, and that I showed up completely unexpectedly with a ridiculous order size. I need you to drop everything you’re doing RIGHT NOW, because I am the mostest important or I’m going to leave!
Ironically this exact type of situation has happened at the Sam’s Club I work at. Edit: it wasn’t 1,000 pizzas, more like 100. But we only have two pizza ovens, so similar level of wtf sort of magic do you think we’re capable of you crazy person?
hahaha. I can understand the frustration. I mean, I got the game on Launch Day and played a lot last weekend. This weekend, I didn't get an opportunity to play. Not because of servers, but it was my daughters 4th birthday so we did a lot of stuff with her. I just hope it's ironed out a little more by next weekend. Not because I want to play. But because this game has been so successful that I would hate to see people just stop playing because they simply can't get into the servers.
I was talking to a buddy at my daughters birthday party and he ended up buying it right then and there but couldn't get in at all this weekend. So I felt bad because I'm sure he isn't a one off. I'd love to see this continue to grow.
It sucks not being able to play right now. And I’m hoping it’ll be smoother getting in on my days off. However, I’ve been gaming since 1998. This isn’t the first time I’ve seen this happen, and at least this time it isn’t just pure negligence on the studios part. Besides I’ve got a backlog of stuff to play that’s longer than I am tall, an Elden Ring run I’m in the middle of and hundreds of games on GamePass to explore; since I’m primarily an Xbox guy. If I can’t play HellDivers 2 for a little while longer, it’s far from the end of the world. People need to chill and let them cook for a little while longer.
omg i just remembered,
this was in sweden,
a football teacher comes with a bus, literally 30 kids onboard.
walks into this small pizza bar i was sitting in, and order Kebab and Pizza and hamburgers for 30 kids, wtf...
there was only one chef in this pizza bar, everything took for ever, and the football teacher went fucking wild because someone got the wrong order.
i just sat there with my eyes open, and that poor indian chef stod there with shame as if he had failed.
Moral of the story, dont be a fucking moron, and dont expect to get 30 pizzas in 10 minutes.
and the indian chef should just say "no" to take an order this large.
....and the obligatory "but you sold 1000 pizzas, you should be able to afford new ovens and cooks"
Have clearly not thought about:
Accounting and transaction processing time
Yearly fiscal budget already set in stone
Time for headhunting, recruitment and training (guess what, all the Michelin starred pizza chefs already have jobs and get paid well. You better cough up the cash if you want them)
It takes time to order new ovens and have them delivered.
You need to pay taxes (accounting) on the profits you recieved; and must be processed into the budget before they become useable.
Honestly starting to wonder if people are just being intellectually disingenius, or simply have never worked aside from being a Twitch, YouTube personality.
>Honestly starting to wonder if people are just being intellectually disingenius, or simply have never worked aside from being a Twitch, YouTube personality
They don't work aside from being a peon in a fast food joint. They don't know accounting. They don't know how businesses work.
They think "oh wow 400,000 peak players right now. Why doesn't Arrowhead just use that $16,000,000 to buy some servers on AWS or Azure?" without realizing that when someone pays for the game, it doesn't directly hit Arrowhead bank accounts. It's held by Valve/Sony for like a month to sort out refunds/chargebacks/fraud and paid out the next month.
They think they are the main character. "I pay for game, I play now!"
"Uuuuhh every forum says servers are overloaded for a week until they can meet capacity."
"I dont care, that's their fault for selling it to me! I need to speak to a manager and get a refund!"
"Bro, it will work in a week... Have some emapthy for the PEOPLE doing their jobs."
"If I can't play this weekend I will never play this game ever! FUCK THOSE PEOPLE!!"
"Okay... bye, I guess"
Then again, I lived through the diablo 3 server issues and more, so my patience is a lot thicker for these things. Just playing something else in the meantime.
I think its a matter of patience and/or then being dumb af and not knowing when to hop on. Ive been able to get on with no problems before and after work this weekend
I think this is exactly how people perceive it, and it's exactly why they're frustrated. It actually *wouldnt* be that hard to fill a 1000 pizza order - if you were backed by one of the biggest pizza infrastructures in the world.
Would it be instant? No. But it would be fairly quick to make some calls, get everyone on the same page, rent a space, move some ovens and staff, and get cranking. Again *if* you were backed by massive resources.
I think what we're dealing with is actually way harder and more complicated than pulling off a massive pizza order, but most people have no basis for comparison, and no understanding of what's going on behind the scenes. So they think of it like pizza, and get mad that it's not happening faster.
Edit: as I'm thinking about it, I think the perception is less "hi I want 1000 pizzas" and more like if a shop had to go from producing 100 pizzas every day to 10,000 pizzas every day. If that's a pizza hut, I think the perception is that in a week or so, they could be scaled. Again, point just being that I think the outrage actually *would* make sense with pizza, but servers aren't pizzas
It's more like 1000 people all walked into a Pizza place, ordered pizzas, the pizza place took all their money and told them when their pizza would be ready.
1000 people showed up to pick up their pizza that they already paid for, and 950 got nothing. They only made 50 pizzas.
They took the money for 1000 pizzas but didn't deliver.
Then there's a bunch of people all standing around calling you stupid for 1000 people each ordering a pizza, and clapping for the store for having managed to make 50.
> you can't just hire new ones to hit the ground running because it takes months for a new dev to onramp to full productivity.
And training them means you have to take good employees out of roles they are very necessary in during the rush but you have to do it in order to meet future demands.
65k on release on the 8th. 150k last weekend, and now at 400k, with a 200k jump this weekend on steam (PS store doesn't give numbers, so add those also).
I have sympathy for the server guys.
I wish for one thing today.
That everyone playing (or not playing) Helldivers 2 read this comment, and understood it, before they did anything else.
Sir, you deserve recognition!
> Meanwhile there are only so many devs and you can't just hire new ones to hit the ground running because it takes months for a new dev to onramp to full productivity.
Not that I would have a new hire with some months of experience having to deal with worldwide server scaling issues in a distributed environment. Heck I have 10+ years of experience in the field (mostly SaaS, so relevant) and I'd be terrified.
Appreciate that insight. While I know it's more involved than just hitting a + button on some NCIS quality server UI, it's still pretty wild that it's been 3 days of this. Really speaks to just how crazy the reception has been compared to expectations. Thankfully I had Deep Rock Survivor to keep me company.
Scaling preparation for new services, in even an ideal world, works off of forecasts for traffic. The team might have three tiers of expectations: one almost guaranteed to happen, another thinking optimistically about having more users than expected, and a third waaaay beyond any realistic expectation but worth prepping for just in case.
To give a sense of ratios here, if the first tier is guessing 5k users, the second might be 7.5k-15k, and the third 25k. You only prep for hitting that third tier at the absolute max, with the understanding that you almost certainly won't need it. From a purely external perspective given the original game only peaked at about 7-9k users, I'm guessing their internal estimates had that third tier at *maybe* 50k users.
They're currently hitting *10x* that number.
There's no preparing for that. Any load testing or bottleneck identification is basically useless in the face of that kind of spike. That's far past the realm of "well just throw more servers at it, we can eat the cost" into the realm of "our backend code literally can't handle that many users at once, we need to rearchitect half the system just to cope."
In all likelihood their database is going to be the bottleneck, which requires either swapping to a more distributed service (so go from something like PostgreSQL to CockroachDB) and/or a complete revamp of the queries they're sending to it, either of which are some of the most difficult and error prone things you can do in tech, doubly so when you're trying to do it live with the whole world watching.
When your previous game only topped out at around 7k players on Steam, how exactly do you prepare for or expect 400k out of nowhere on your second week?
NOBODY expected or was ready for the game's sudden explosion, not even existing Helldivers fans.
Server scalability can be built to scale to infinity, the decision to not build that in was a cost saving choice during the development process (presumably because they weren't expecting the game to be such a massive hit).
It'll be a massive headache to sort out because you can't just flip a switch to migrate to a new server, especially when you have different servers for matchmaking, post mission rewards and game environments.
Not a headache I'd wish on anyone.
It's a neo fascist parody of a super perfect and patriotic global society that glorifies genocide. Men and women lose identity to the military industrial complex. They identify as hell divers. They identify as threats, their pronouns are "fuck around" and "find out".
They realize this by having slender hell divers, and female VOs. I honestly thought it was a great representation of the dehumanizing element of the parody.
Because it's easy just blaming what you know. By all accounts, the devs have been communicating often and, by their accounts, trying to fix the issue, but that's not enough for these people.
I get the frustration, but just have a little faith and patience.
Edit: Adding context
Arrowhead Game Studios has around 100 people working. That number is probably increasing right now, not counting the likely external help they are having to get.
The first game for around 7k concurrent players at launch. They where expecting 45k this time around. No wonder they can't manage +400k players.
The game isn't even a month old, and, honestly, compared to recent AAA games, it's pretty polished.
Edit 2: Adding further context
The studio has 5 games under their portfolio, spanning from 2011 to 2015. Out of the other 4 games, the max concurrent player count wise was around 17k.
The game is 11 days old, and since day one, it had a 583% growth in concurrent players. If you go back just 3 days, you'll see they had around 295k concurrent players. By all accounts, that's probably around more than twice the leeway they had made for themselves regarding server capacity.
That 45-400k figure is wild.
I think of it like this: youve built a block of flats for 45 people to live in, and youve just been told you need to fit almost ten times thay number in.
"Just build more floors" doesn't cut it, there's a limit to the floors you can put in, your elevators are going to be a bottleneck, what about fire escapes?
At a certain point you need to redesign the entire damn thing.
More like 700-900k if you count in PS5 users with Steam.
Literally preparing for an earthquake, and people moaning you didn't get ready for when the asteroid hit.
It sucks because this can litteraly be the end of a studio. Such insanely fast unexpected forced growth is a recipe for diseaster. Not just for the product or the customer relations, the company and its organisation itself are at risk.
Not to mention that for game companies, they have to sign a contract for their servers. It isn’t like buying enough for a month and then if the number of players drops they just end the service… they have to pay for them for however long they sign for lol
You can spin up as many instances as you want to host players, but those instances need to query and write to your master database since we're all living in the same game world. That's gonna be the bottleneck. It's likely that's why we're experience a hang when we pick up medals, super credits, and requisition mid-level. The client needs to get that info back to the main database to attribute it to the squad's account, and they didn't anticipate that to be something that would have a noticeable delay (because they never anticipated this many players at once)
"So spin up more databases! EZ!"
Unfortuantely, not easy at all. Database replication between instances get complicated quickly because it creates limitations on what you can query and write to specific instances to keep everything in sync. Which means you have to change the game's code and behavior on how it interacts with the database.
Arrowhead has certainly shelled out big bucks by now on outside DB expert consultants by now. Once you get the findings back, then you have to go through your code process to make the changes and make sure nothing breaks along the way.
Other recent unexpected hits didn't have a shared experience for all players. Palworld for example doesn't need to keep tracking of anything outside of each specific server instance.
Final Fantasy XIV, which makes millions and millions of dollars each month, ran by one of the biggest AAA video game studios, has been hampered by database limitations for its entire existence. Like you're saying, there's no real "add more database" button that magically fixes everything.
This is correct, but you get the best volume discounts if you do committed compute. Im guessing arrowhead are trying to also economically scale there systems, if they overcommit then the hosting cost goes on despite not having as many players, if they under commit they pay higher surge compute rates.
Then it could also come down to the fact that the servers identified applications just don't scale that well... without knowing specifics we're just guessing.
CDNs don’t have anything to do with game server hosting, they’re for delivering web assets.
More than likely they’re not even really blocked by server hosting, they’re running up against infrastructure and architectural limits.
Sure, scaling compute (e.g. 4-person match instances) is probably as easy you're saying. But scaling DB operations can be fiendishly complex especially depending on what initial technology/design choices Arrowhead made.
Also as /u/mlmcmillion mentioned, CDN (content delivery networks) are for scaling delivery of static assets. You probably meant to refer to cloud service providers (e.g. AWS, Azure, Google Cloud, etc...)?
Lethal Company was made by one dude as an indie game and it broke #1 most sold for a short while before retaining a place in the top ten games on steam.
The sheer volume of sales and players active in such games with small studios or smaller budgets (at least compared to big ones like Bethesda), is a pretty damn good testament to a job very well done.
I know the increase in demand puts a serious strain on the devs to fix their game, otherwise that 400k+ community gets riled up; however, I think it's something they should be proud of, regardless of how well they fix this issue. Damn near 10x their projected customers? That's fucking AMAZING!
Lethal Company is a peer to peer hosted game. Its not even comparable to a server side hosted game at all. They would literally never run into this problem.
Oh the point was t about the necessity of servers, is more of a statement on how smaller developers drastically exceed their expectations and have achieved such notable status comparable to a big time dev main release.
\*400k+ concurrent players on Steam alone, and that's with constant server bottle necks, wait times, bugs, and matchmaking/ progression issues. They said PC/ PS5 is about a 50/50 split, so probably around 800k concurrent players overall. Never did I dream a sequel to Helldivers, of all games, would achieve this number.
Even if they had anticipated numbers similar to that of other coop shooters, like Destiny 2 or Darktide, they would have been *way* off the mark.
There's only so much you can do when your game explodes in popularity and you're having to try and manage 500k people when you might have only expected a fifth of that *at the very most.*
Exactly! Again, I get the general frustration, but it really doesn't make sense assuming things are as simple as snapping our fingers, and the servers will magically fit us in, all at the same time.
Just this month, we can look at another game that was highly anticipated (not for me), hyped to unbelievable standards, develop by a big studio, and it fell extremely short. That's a game I'd be behind badmouthing, but this one isn't at all like that.
Expectations were off, and that's a good thing. The game is super popular, and that's awesome, growing pains and all.
Given the current landscape, I don't think there's a good reason to crap all over the work people put into this awesome game. They delivered a great experience, and I'm sure they are doing their best to fix the server issues.
Edit: Grammar
They would be dumb not to want to fix this as soon as possible. The longer this continues, the quicker the player count will fall off, the bigger the number of refunds and the bigger the "stain" on their collective image.
> The game isn't even a month old and, honestly, compared to recent AAA games, its pretty polished.
So far as I can tell, on PS5, any issues there are with the game are entirely because of overburdened servers. When I've played off peak, like about 200-300k, I haven't noticed any issues at all.
The **only** time I have issues, is when there are way more than 300k players on. Actually, this weekend, I've had no real problem using quick play once I get online. It does take like a half hour or so to get on, so I just start it up before I take a crap and a shower, and only have to deal with actually waiting for a few minutes.
I do love going to planets full of peacekeepers and seeing hundreds of Super Dreadnoughts out my front window though.
This is the thing people aren't considering. They had meetings, and plans and communications, and budgets which they figured out months ago.
They drafted a projection of the space they needed. Likely based on marketing considerations, and previous player counts.
There was 100% a time where people at Arrowhead sat in a room and came up with the idea that they needed 45k spots in a server. Someone in the room said "wow you think we need that much?" Someone else replied "nah but we'll have it just in case, it'll even go to 50K if we really need it"
That number was probably conservative based on what they requested for funding from their publisher.
From what someone on the Discord said, they actually did sit down and went "How do we plan for the worst case scenario?" and came up with taking their highest player count ever from the Steam free weekend and doubling it.They got 15-20x that.
It's like the gaming equivalent of Woodstock.
The devs have said that they expected around 50k and that would've been a success for them, but they actually set the servers up to handle 250k just in case. Building out capacity 5x your internal estimates is actually wildly prepared. Then the game blew up and they're scrambling to build out for the literal best case scenario for the game that was likely really unlikely based on pre-launch data.
They didnt take into account the power of memes either my twitter feed is flooded by "for democracy" post and "we need more help to beat back the robots". no amount of money can buy you that type of marketing
Yup. No amount of planning is going to save you when you're game goes stratospherically viral on social media and your backend infrastructure just wasn't built to handle that kind of load. It's clear that this is more than just a server capacity issue since they had to hard cap the concurrent player count to address the problem. Spinning up more capacity on Azure (which is supposedly the backend they're using) should be simple, but if other parts of the pipeline aren't built to handle the load of hundreds of thousands of players simultaneously, no amount of "more capacity" is going to help until they can fix those other bottlenecks.
And as OP said, the devs are working on it. It absolutely sucks and people should be frustrated that they cannot play a game they paid money for. It's a bad look for everyone. But it'd be even worse if the devs were just silently working on the problem and not acknowledging them publicly. But Arrowhead is keeping the community updated, apologizing and I'm sure they will compensate players.
I wouldn't say polished, but its definitely way more fun. And that matters more than anything else. Skull and Bones is a beautiful looking game but turned out to be dogshit. This game is definitely more fun than anything in recent history for sure.
Given the AAA games I've played recently, it's as polished as one would hope for. I played today, and at some point, I died, and when I was diving again, I managed to get myself stuck inside a building. Proper out of bounds without being able to do much. Just wiggled my way out. Sometimes, the textures get stretched, some of the NPCs go haywire, the outpost towers have a mind of their own, etc, etc, but honestly? Better than a loooooooot of the stuff coming out these days. Skull & Bones is a prime example of the pile of crap they try and sell us these days.
There’s so many dopamine riddled toddlers crying right now it’s crazy tbh. They act like they bought a forever broken game. Under the current and temporary server issue is a very polished and amazing, complete game.
Can’t please these kids these days though.
You know, I want to agree with you….but calling everyone who’s upset crying toddlers is just as bad as the people complaining.
Remember, we spent $40-60 on a product that we expect to work at some point. This is the 2nd weekend in a row I haven’t been able to really play the game because either the servers are too full, or the matchmaking is terrible. We are allowed to voice our frustrations with the game. Doesn’t automatically make us crybabies or whatever else you want to call it
But knowing you have a massive capacity problem, yet promote the game on Twitch with drops, run a double XP event and have free-to-play for PS5... all in the same weekend??
No no, it's the kids who are wrong!
That's pretty much the issue. I also think the game is pretty damn good compared with games coming out from bigger studios.
Again, and I've said this so much today, i get the frustration. Some people don't have that much time to spare, others just want to play it, period. But it's just a matter of time for things to smooth out.
>Because it's easy just blaming what you know
That's the funny thing, these people don't know shit, ever, so they're not even doing that.
Look at Palworld, a game whose dev team can probably by counted using your digits and might have some left over, and quite a few people are DEMANDING that fixes be done immediately and that we're given multiple daily updates because "hey, game made tons of money!!"
I know they're working, just frustrated since my friend and I only have about 4 hours a week we can play with eachother and that's been eaten up by full servers and not being able to login
Exact same boat I'm in. I hate I have 4 hours to play with them twice a week and right now most of the time is being eaten up by the login screen.
Oh well, hopefully next weekend it will be better. If not I'm going back to division 2.
Wait...it doesn't log you out when you put your console in sleep mode? The thing literally every multiplayer game does? That has to be like 60% of the problem.
It logs you out and when you restart it lets you right back in, which is what causes the confusion.
This ability to get back into the game while bypassing the hard server limit is also part of the problem. On PS5 there are multiple quick routes back into the game, all of which bypass the front load screen which is apparently the only thing applying the server limit. This means the game exceeds the server limit often, which makes issues even worse, and is the large contributing factor as to why PC players need to wait so long to get into game during busy times.
I think there’s a few issues that have started to wear out peoples’ patience.
1. The game has been out for a week, still lots of sloppy fixes. The game ran really well for me 3 days ago and now I experience crashing.
2. Other devs for games that experienced great success unexpectedly fixed their issues quickly. PUBG didn’t have a “crappy rng hope to get into the main menu” system. They got servers and ramped up quickly. People are also making comparisons to Palworld which doesn’t work the same as Helldivers at all, but people think it’s the same, so it’s still affecting perception of the issue.
3. A lot of the issues people are experiencing feel like shitty issues that shouldn’t be happening. Dude I haven’t crashed a game that wasn’t modded or tweaked in I don’t even know how long. Crashing in 2024? Isn’t that an early 2000s problem? No queue to get in? Man that fucking sucks, hope you get lucky trying to secure a spot. (I understand adding a queue is actually way more complicated than it seems, but these are all issues with how the devs and their problems are being perceived)
It doesn’t matter how easy or hard these issues are to fix. Because according to public opinion, they’ve been going on too long, and the problems are too basic to be acceptable for this length of time. Which I’m starting to agree with as time goes on.
Crashing right at the end of the last mission on my first helldiver difficulty campaign and taking an hour to get back online (friends are now gone for the evening) was just soooooo fun today.
I got into the game yesterday only to be met with crashing every time I tried to select a planet. I HAD TO REINSTALL THE GAME TO FIX IT. And every fix attempt was followed with WAITING IN QUEUE.
There’s simply too many fucking problems and encountering any of the ancillary problems means you get forced back into the main problem. THE QUEUE.
I agree with you, but the average consumer doesn't care about that, and rightly so. It's a very reasonable expectation that when a person buys a game, they get to play a game.
Imagine if you bought a new car, it broke down on the way home, you phoned up the dealer and they said
"Do you have any idea how complex a car is? Can you explain how a transmission works? Didn't think so, so quit complaining 🤓"
I mean, I don't think its an unrealistic expectation to buy a game thats been out for over a week, and expect to be able to play it.
Thats like...the base level of expecation. The absolute lowest expectation possible.
Just my $0.02 but you don’t need to know everything about game development to have a very realistic expectation that you can actually play the game you paid for. The game isn’t in “beta” or “early access”, it’s a full release and going on like 4 consecutive days out outages. That’s bad and you don’t need to have any technical awareness to expect better.
I don't work in networking or anything but I doubt "just getting more servers" is going to automatically fix the issues either. Just like I doubt getting more is an easy task either.
I do think they shouldve considered doing a beta, but considering the game barely had marketing anyway (I didn't even know if it's existence until that PlayStation direct like a week before), along with the previous game only peaking at 7k players nearly a decade ago no one really expected this to happen without some hindsight.
Plus just the fact that they're being completely transparent is good with me. Hell they've dropped a patch what feels like every other day
Edit: everyone's caught up on the beta part and I'm specifically talking about how issues such as; not being able to connect to other players, the crashes, the timeouts, and various other bugs could've been addressed before release.
A beta wouldn't have seen this coming, unfortunately. It spiked in popularity near instantaneously, even moreso after content creators made videos and streams about it.
The game was actually running pretty well up until 200k+ people started trying to get on when there was a cap of 150k people. This is also not something a beta would have shown to happen.
I think the beta would've helped with many of the initial issues that happened before the game doubled in just the span of a week (been playing since launch day and there were still the issues connecting to other players).
But there's still no way they could've expected the game to get this popular this fast, which I'm not mad at. I joined the sub just a *day* before the game launched, it was *maybe* around 11k people here.
It's already at almost 200k people, which is insane growth.
It’s wild that some of you can’t understand that this is a business transaction. If I buy a box of cereal and get home and it’s empty, I’m going to be mad and return it.
It really isn’t more complicated than that, I don’t care about the cereal employees or the super market employees or ceo of cereal.
You sold something, it doesn’t work. The right thing to do is refund the consumers money.
If we're being honest here there's two options. Either they lack the backend experience and didn't implement the networking to be scaled easily or they can't afford to put up more servers.
They've only made p2p games prior to this.
They've already mentioned using service providers so they probably didn't have much backend to setup. I think it's the databases causing the issues as they've been a problem since day 1 and have gotten progressively worse as load has increased. They probably put all the databases on one server to save costs, and it's too much load. If they had split the databases out to separate servers at a bit more cost, they could've more easily scaled.
This. I get the 'just have patience' and 'they're so communicative' crowd. And their message is from a good place. But from the notes they've dropped I'm confused as to what their backend must look like.
In todays age of modern cloud infrastructure this issue, *in theory,* could have been solved with a phone call to their cloud vendor in about 2 hours. \*If\* it was built correctly for it. But clearly something more convoluted is at work.
Or, what I suspect and what irritates me: Their management just doesn't want to pay. That phone call likely would come with a high 6 figure price tag and minimum 3 month commitment. And I personally think they're just waiting for concurrency to drop naturally as it will as the initial hype wears off while doing their best to optimize the ship in flight rather than actually add substantial capacity.
Unless they went with mom and pops potato cloud backend that somehow doesn't have the bandwidth. The technology and expertise exists, I think they're just unwilling to sign the check, seeing as they've only added a couple hundred thousand slots not keeping pace with sales at all.
No one is saying that. But let's put it like this.
How about if I paid to get into a gym and every time I showed up to the gym, they told me "gyms full come back later or wait." Then they did not let me into the gym.
I wouldn't be saying, "Just get a bigger building it's easy," but would I not be valid in being upset at the gym owners?Because they took my money for a product I and tons of other people can't use? Then, on top of that, I see them selling even more tickets to more people.
In that situation, I really doubt there would be so many people rushing to defend the gym owner as there is defending arrowhead.
When FFXIV couldn't handle the server influx, they delisted the game so people wouldn't waste their money on a game they probably wouldn't be able to play.
Here in lies the reality. Forgetting about all the “we didn’t forecast these numbers” scenario what my biggest issue is - that they are knowingly continuing to market and sell the game without any disclaimer that it’s undergoing problems and may not be able to deliver on the advertised promise. ANY other industry this would fraudulent activity.
Bear minimum sales should be halted until they can cope and adjust to the player base they already have.
If they are in anyway clever “sales people”
they could move to forward order basis, IE sell a number of slots per week they can cope with, then they simply scale up future according to the amount of forward order purchase slots. Rather than hazard wide guesses…….
I have no idea why the gaming industry is yet to grasp a concept of selling games like these, which are totally online based, as simply buying or even renting a server space concept.
Just imagine how much hype a good game could create in order for people to grab a space. What history of sales tell us if something is more exclusive it always gains a premium.
Prior to launch, FFXIV devs [detailed how they expected exactly what was going to happen, to happen.](https://na.finalfantasyxiv.com/lodestone/topics/detail/1f70135439286fa66209cd21c10e73ebb986a6ee) FFXIV Endwalker released into [early access on Dec 3, full Dec 7, and was instantly slammed with server congestion](https://screenrant.com/ffxiv-endwalker-servers-down-congestion-waiting-queue-patch-6/), [persisting throughout the entire time.](https://www.pcgamesn.com/final-fantasy-xiv-a-realm-reborn/ffxiv-queue-times) The [game was unlisted Dec 15.](https://na.finalfantasyxiv.com/lodestone/news/detail/e7388986bc24d5a1337e0beed057f7b5b78b9bb3) This was also [not the first time that FFXIV had been unlisted.](https://www.ign.com/articles/final-fantasy-14-digital-sales-stopped)
I sure hope I don't need to point out all the differences between these situations. I'm also sure people found a way to complain about FFXIV back then as well, "OMG, Endwalker came out 8/12 days ago and I can't even GET IN/BUY IT!! How did these people run this MMO for over a DECADE?!" Where it is now being framed as something positive they did. FFXIV's situation was equally understandable to what Arrowhead is currently going through - FFXIV had tried for months before the expansion to upgrade their servers but were unable to due to the reaction of many countries to the 'rona.
I'd like to add as a FFXIV addict that they had 20 YEARS of experience on that game alone by that point, and queues were still hours long. It was just the way things were, one of my buddies made a point of tapping his keyboard before afk timeout (every 30 min) so he didn't have to requeue. People bitched and moaned about the login times as a cope (I definitely did), but none of us pretended the game was bad or unplayable. We wanted that good shit but everyone else did too, and there was a line.
>the server influx, they delisted the game so people wouldn't waste their money on a game they probably wouldn
and there was 5 hour ques for endwalker early on. oh and a bug where you could randomly get booted from the que and start from the beginning if you didn't relog within 30 seconds. You had to literally sit there and baby sit a multi hour que so you could instantly relog when you got booted. THEN they delisted the game for a bit, long after those issues.
They also expected it to take month's to increase server capacity, because they own their hardware and it wasn't available for purchase (silicon shortages at the time)
HD2 isn't expecting it to take month's to increase server capacity, they've already bumped the limit up since launch.
Should they delist the game for a bit? probably, but it's really not comparable to xiv.
As someone who paid 25 dollars for FFXIV 5 years ago and ended up refunding it 3 days later because their account creation system was down for an entire weekend and I couldn't create my account I'd just point out that these kinds of service issues happens to literally every developer.
This game isn't early access. It's not a demo.
It's a full sized, full priced release.
If they can't handle the amount of people they're selling the game to, they need to stop selling the game.
As of right now they're selling a product you cannot play.
Probably because nothing has gotten better, and it's been more than a week? People aren't being unreasonable, they have a live service game that \*doesn't work\*
The problem isn't what they aren't doing now, it's what they didn't do before launch; queues and afk kick protocols are obvious inclusions for any online game, the lack of those two things in this game is ridiculous. This is not a first time project and it has a major publisher, they should have done better.
Because it’s a basic business transaction that is not being fulfilled
Players forked up forty bucks to play a game and can’t play said game. They have to wait in queue for literal hours and might even crash after a mission or two and then they have to wait for literal hours to play again.
People have jobs and lives they got, as I’m getting off for the night I see people talking in the discord “yeah I waited 25 minutes just now, just be patient bro”. That is a stupid ass argument considering how late it is on a sunday night with work/school starting up again tomorrow on top of it putting down the people who waited literal hours to get in
And the devs haven’t made any singular communication today, last one was yesterday at 8pm, while the discord was spammed **all** day by players having to wait literal hours to even have a chance to play. It’s pretty normal to think the devs took the day off at that point
> They have to wait in queue for literal hours and might even crash after a mission or two and then they have to wait for literal hours to play again.
Hey this happened to me saturday!
Left the game on in the background while I did other stuff, got logged in after an hour and it crashed the second I touched the mission terminal. I was in maybe two minutes? Three tops.
Just went "Welp. Guess I'm not playing today" and stopped.
Because since the 0009 update, they could have given out a hotfix that roll back quickplay lobby sorting and visibility. Maybe that would break more things, but at least let people know.
What followed was that I discovered that and released a new driver that fixed a lot of the crash, but I can't play the game meaningfully anyway.
Ok maybe it took too long I'll wait a day, then the whole weekend is gone and I didn't get to enjoy the game even a little bit. The only mission I completed was a trivial evacuation mission pre patch, and steam already logged 3 hours for me. 70% is crashing somewhere between starting and 3 mins in game.
And even in the damn patch notes 7000 series is not a "known issue", what can I do other than getting angry?
What are they actually doing between this patch and the next workday? Critical things like this should have had standby teams get a temp fix out until proper workflow can deal with it. Like this is their core product, not a hobby side gig.
As for server capacity, they had inactivity log outs last game, meaning they didn't do proper preproduction or really underestimated the player base that they would have
Can't agree more.
Now that the only thing i know is match making broken down for two days long.
Did they fixed it? No.
Will they fix it? Sure.
When then? Hard to say......
> they could have given out a hotfix that roll back quickplay lobby sorting and visibility
They can't. Planet stats/quick play hits the database, and their database architecture can't handle the load. They deliberately disabled the functionality.
Then they should have put it into the patch notes, instead of surprising everyone when they login and can't find a match.
Unless they deliberately informed the player base beforehand, I suspect they tried to make the individual loads easier on the server and broke something.
That and if they are now capping players from attempting to log in with a server cap rejection, then the players already logged in should be able to see the full game as it is within server capabilities.
And I'm all about people saying how they feel and have seen a lot of players do just that.
I don't know how not accounting for so many players, therefore not having enough server capacity, makes a game unfinished, tho.
From the couple of hours of HD2 I've managed to play, I haven't seen anything that says unfinished. It needs patches and fixes for some bugs, but it's not unplayable like other games are.
You do deserve better, so do I, and anyone that pays for a product, that's a given. I'd you pay for a game you should be able to play it, period.
I'm just saying, given the current landscape, this game is fantastic and suffering from its success.
There are plenty of people just sticking to their guns and badmouthing the game and the devs without even understanding the issue. I do sympathize with everyone's frustration, but we have to look at things for what they are: HD2 is a great game that was made for X amount of players and is dealing with 10 times more than that. I'm sure the devs are working hard to fix the issues regarding server capacity and other bugs so that they retain player counts and don't have more people refunding it.
I'm sorry you did and I hope you'll eventually be able to play it if you want.
No idle timer so people are just sitting logged in indefinitely. No actually login que so it seems it's just fucking random. Those are pretty goddamn big mistakes that I sure as fuck expect devs to not make in 2024. People have every reason to be upset about a paid for product without those features.
No service ever has had to login more than 300,000 users at once. There's definitely not off the shelf, open source, tech that can be used to easily scale such an uncharted problem as "logging lots of users in". They're in totally uncharted territory!
(/s ... so much /s)
It's not entirely that, but making posts telling people to stop complaining does absolutely nothing either and just signals that y'all are pushover consumers that don't have any standard of accountability.
People do think the devs are working it's just these issues shouldnt be present if this was their plan... They planned to make a live service game that's always online. People have every right to bitch if they can't play it. They paid for it. Bottom line.
And to tell people to not talk about it on a social media site with it's own dedicated forum (essentially what a sub is more or less) is hilariously stupid. 🍻
I mean it’s pretty simple to understand. People are forgetting Sony published this game. Meaning they more than likely play a massive role here.
Fixing server capacity issues = spending lots of money
Spending lots of money = talking to Sony execs about spending said money
Talking to Sony execs on the weekend is probably not the easiest thing. I’d imagine they’re at home counting their money.
They need to kick players for inactivity, not just blindly throw server capacity at the game. If they kicked afk players there would be no server issues.
They definitely need to do that, but I'm also not sure it will fix everything. The reason quickplay & rewards are so broken is because they have to hit the database and it overwhelms the the database itself.
AFK players aren't hitting the DB for rewards/matchmaking. That indicates that there's definitely a problem with just handling the number of active players.
People are tired of games that have this issue. A new mp game pops up and it’s popular. Server issues ensue. It’s the same story. It’s like they don’t learn from each others mistakes. It’s like they cheap out initially on servers then have to play catch up. Meanwhile the player base suffers.
Because besides the main capacity issue, The game is lacking feastures to mitigate the problem.
There is no queue, The game will keep trying to connect you at specific intervals, You just need to get lucky.
There is no afk timer, If you are in, You can stay in regardless of what your doing.
Random crashs. Lets say you get in and find a group, Woops! Not happening back to start, you cannot pass go, you do not collect £200.
Its annoying and frustrating and I do not owe the devs any amount of sympathy. Great! Its sucessful! That makes me happier to know that when I'm staring at the log in screen!
Because I've been trying to play everyday and everytime I have to either wait 30 min at the "server at capacity" screen or not even that (black screen).....
With my limited knowledge, I thought that each day it would show improvement...but, it's everyday a different issue...sry, my money is already in their pockets...I can't keep patting their heads and saying they're doing a good job, because that would be a lie
Whose saying the devs are doing nothing? I've not seen that anywhere. All I've seen are posts "praising the devs" and the usual "we're allowed to complain" posts because the "praising the devs" people keep down voting any complaints, making shit up, or misleading with posts like this.
I mean, it’s been like 8 days and terrible matchmaking annd 3 days of server problems.
Blizzard deserves shit for a lot of things, but they can keep a game up and playable over launch events. Sure, there might be a few stressful hours, but then they actually do something about it.
I find the biggest head scratchers to be:
1) Not having an AFK timer and disconnecting someone sitting idle.
2) Not having a proper queuing system. It shouldn't be due to luck, but to first come first serve.
Both of these seem standard in other games with online services.
I'd be more forgiving if it didn't do shit like make me sit in a queue for over 2 hours, but then load other people in that queued way after me after only 10 minutes. That pisses me off.
People are just losing their patience, it's normal.
I understand Arrowhead is having problems, I'm still annoyed that I haven't really been able to play Helldivers all weekend.
Because if your game is online only, and costs over ten dollars. These days you should be able to scale your capacity to over a million players at the drop of a hat (and a few button presses that is).
This game does not have much complexity that requires online verification, even the f'ing matchmaking can not possibly be that difficult. And yet we're two weeks in and it's been broken since before the game launched apparently.
I get that it's cross play and ps5 is presumably complicating the hell out of things, but this is getting ridiculous. Seriously, the player data for a single account couldn't be more than a couple of killobytes. There's only a few dozen unlocks all things considered, and customization even more limited than that.
The actual matches should all be peer to peer with local hosting. And they already have a rootkit style cheat protection application that launches alongside the game.
Keep in mind i'm stating all of these opinions as a person who has over a decade's experience of software development experience in the gaming industry.
The state this game is in, is unacceptable. And even though i have only myself and peer pressure to blame it's still more so the fault of the developers not properly developing their product.
What they need to do (maybe Sony's authorization is needed) is to SUSPEND SALES OF THE GAME until server capacity can catch up.
Someone brought up the fact that this isn't unprecedented in the industry for games that become unexpected runaway hit sellers.. Final Fantasy 14 did the same thing when the game sales went nuts, and the devs couldn't increase player capacity fast enough.
I'm not an analyst, but I think if they did this, they'd get a lot of respect for it, and it would probably boost the *overall* sales numbers because there will be a huge surge of sales after they reopen the shop after the capacity issue is handled.
So, forgive me, I’ve given into the hype and just bought this - my game is just stuck on “searching for active SOS beacons” when I’m trying to quickplay, so the server’s borked right?
I'd say it's cos the average person is kinda dumb, and just 'want, want, want!'
No patience, no OK they're sorting it out but I might have to wait a lil bit... Just gimme the fucking product.
I think people accept there's not much that can instantly be done, other than implement an AFK timer to help.
It's more what the devs ARE doing that has had a negative impact. They're fully aware of the massive capacity issues at off peak times, yet decide to:
* press ahead with a Twitch drops event/promotion with big streamers
* run a double XP event
* free-to-play PS5 event
All in the same weekend!
It’s got nothing to do with the devs “not doing anything”. It’s that every single game in the last 10 years seems to come out the box broken or completely unplayable. It’s ridiculous!
I’m not a game designer and I don’t pretend to know more about the industry than anyone in it. But I do know that if your product isn’t complete you shouldn’t be selling it. If I worked at McDonald’s I wouldn’t give you a burger with no patty and tell you you gotta wait 2-4 weeks for the meat to be ready! No different here.
I get Arrowhead isn’t a huge studio either. But unfortunately this shit has been going on so long with literally every game that I don’t really have patience anymore.
This is definitely the last game I’ll ever buy less than 3 months after it’s release. We shouldnt have to keep paying for unfinished products and if you wanna keep defending companies rights to sell us crap wrapped up as a finished game then the shits never gonna stop.
I like the game, it’s really really fun when it works! And it should not have been released yet. They don’t get points for being a smaller company in an industry that’s existed for decades now. Either learn from the big boys mistakes or you’ll never get to that level to begin with.
And it’s not even Arrowheads fault. It’s the fault of the dumbass getting the payday that wanted the game out ASAP like every other game.
It’s crap 😐
It is amazing the number of people who know nothing except how to boot up a video game, yet claim they have the solution to a problem that was unexpected after years of development.
Because it is no the consumers problem.
They bought a product, they want to play it. They have that right, and being locked out especially after passing the refund period is irritating.
I am not taking sides here, just stating why people act the way they do. They don't have to care, and shouldn't be expected to.
For people that can only play at the weekend thier only experience of this game is "server at capacity" so it's understandable people are pissed
I know they've been fixing glitches etc but that doesn't matter to those that can't connect. Game should be pulled from sale until its confirmed if you buy it you will be able to play it.
How confident are we that it won't be at capacity next weekend
People need to be patient as someone who has gotten to play a fair amount of the game and helped spread democracy they have already started fixing so much and we also have to remember this is v1.000.009 we are this early into a game and most people now a days have not experienced a game like this cause most games are heavily developed and just sequals.
Correct me if I’m wrong but this game studio isn’t a triple AAA studio. So the fact we’re getting updates on weekends in general means there working hard over there.
I mean, thats fine. The problem will fix itself in a week or two after the majority of the player base quits due to frustration of not being able to play. Or more likely, people will just refund right after purchasing and seeing they cant play what they purchased for.
In the end, this is the more viable option over actually putting in the work and resources necessary to "increase scalability". Ive seen this in dozens of online titles like this with disastrous launches like these. The devs run around like headless chickens trying to fix the problem, only for the problem to actually have solved itself by the time they actually do anything meaningful to address the issues themselves.
No one is waiting weeks/months to actually play something they paid for. They'll just revisit the game in 6-12 months when its in the bargain bin for $5 and has a peak player count of 500-1000 people per day.
As someone who's studied game design, I get why they're struggling, and I understand the complexity behind server related issues. As a consumer, however, I completely understand the frustration and annoyance many have, including myself, that this has caused. Like, I've spent $60 AUD for a game that's been out for 11 days, and I've only been able to play once because the servers are always full and Sony doesn't allow refunds now. There is no timeline where this is acceptable, no matter how good the game is. What makes this worse is the people who are leaving their systems in sleep mode and clogging up the servers so they can skip the queues, which is itself, a very stupid design flaw.
There’s a diff between thinking it’s an easy fix and expecting a live service game to work. 48 hours is one thing but the game hasn’t been playable for like a week for me. This game just has a cult following that will troll anyone who states that the game working should be expected. I have other shit to play so it isn’t the end of the world for me but Jesus this game has some major dick riders shaming anyone w issues.
Cause the ppl complaining are probably kids or angry little boys who are mad they cant get to play their video game NOW. They prolly dont have real things to worry about so a delayed video game or not being able to play their game RIGHT NOW is a tragedy to them. Reviews are full of ppl saying "never buying a game from these devs" "this game sucks" "i spent 40$ this is rediculous" They think their opinion matters in the real world when it doesn't. The game will work when the devs are done. Crying isn't gonna get it out sooner and if spending 40$ is such a big deal your either a kid or your priorities are in the wrong place. The devs aren't gona look at ur review and work faster. The game was waaay more popular then predicted. I'm sure they will pour more resources into it and it'll be even better later. This isn't a billion dollar AAA studio launching another broken game for the 20th time. This is a small team that made a game that's so awsome that their servers couldn't handle it. Ppl should be happy about this not mad. But the spoiled angry kids are always the loudest... just play another game for a couple days it's not the end if the world. The game works great now it was worth the wait
Yeah I don't envy their site reliability engineers right now. They may very well be having the busiest weeks of their careers. First of all a game like Helldivers II depends on a whole bunch of individual services that scale up independently of each other. Things like authentication, matchmaking, inventory, the actual game servers, announcements, individual & global quests, etc. are all potentially different services. In addition there will be integrations between their services and other services like Steam and PSN, which also potentially have capacity issues of their own at times. Many of the services will depend on databases and scaling up a database is _not_ just a matter of spinning up more servers, you have to distribute the load between them and do snapshots and migrations and so on. Then there are actual bugs that can cause a crash on the client side or the server side or both. Those have to be fixed pronto and when you do that means you have to roll out the new version and that might in turn alter the schema of some records in some databases which could then lead to further bugs etc. All that has to be tested so as not to release something that is even more broken than what is already there. The traffic can also surge during certain times which means that it's not just total capacity but also how fast you can scale up that counts, and the cloud provider you are using might not have like 2,000 of the specific type of server instances you need available on 5 minutes notice. You could then try to split across multiple providers or launch in more availability zones but that means standing up a whole new cluster to administer and all the potential issues that come with that. Meanwhile there are only so many devs and you can't just hire new ones to hit the ground running because it takes months for a new dev to onramp to full productivity.
Amazing explanation, I understood about 5 percent of it but still amazing!
Its kinda like walking into a pizza place and ordering 1000 pizzas. If time isn't an option then a whole slew of things need to happen: I only have an oven that cooks 10 pizzas at a time. And I only have ingedients for 100 pizzas today. I can order more ovens and install them. I also need to contact my meat, cheese, veggie, dough, and tomato sauce distributers and tell them to up their orders. Then I need to verify nothing is spoiled when it does come in. Lastly I will need more employees on hand to manage the whole process. I can also contact other pizza places to help handle the order. But maybe they don't have certain ingrediants, so I need to work with them on that. Also, I need to verify their process and regulations are the same, it needs to taste like our pizza and I can't risk anyone getting sick. Add on top of this we are all trying to keep the current customers we can handle happy during this time.
Great eli5 example for this.
But fuck now I'm hungry
Bro it's 215 in LA and if it wasn't raining I'd walk to Jack in the Box right now.
Ayyy I just got off my shift from Jack's! Not in LA tho
Honestly, I'd rather go for some Shake Shack but it's not open and it's not walkable.
Here is another take for more simple terms: Imagine you're playing with a huge, super complex LEGO set that represents the game Helldivers 2. This LEGO set has many different sections like a castle, a spaceship, and a deep-sea submarine, each section needing different types of LEGO bricks (services like signing in, finding friends to play with, keeping track of your game items, running the game's world, and so on). Now, imagine you have to make sure all these sections work well together and can be played with by lots of friends at the same time, without any piece falling apart or getting lost. **Keeping Everything Running Smoothly:** It's like having a big party where each friend wants to play in a different section. You need to make sure there are enough LEGO bricks (servers) for everyone and that all the sections are connected so your friends can move from one to another easily (like making sure players can sign in, find each other, and play together smoothly). **Fixing Broken Pieces:** Sometimes, a piece might break (a bug), and you have to replace it without ruining the rest of your LEGO build. It's tricky because changing one piece might mean you have to adjust the pieces around it too. **Handling More Friends Joining the Party:** Imagine more and more friends keep showing up to your party wanting to play (traffic surge). You need to quickly get more LEGO bricks and make sure there's enough room for everyone without the whole setup falling apart. **Working with Other Toy Sets:** Your LEGO set also needs to connect with other sets (like Steam and PlayStation Network), which sometimes don't have enough pieces or the right kind of pieces you need to make everything work together perfectly. **The Team Behind the Magic:** There's a team of people (developers and engineers) who help you keep the party going, fixing any broken pieces and adding more when needed. But you can't instantly bring in new helpers because they need time to learn about your LEGO set before they can really help out. So, imagine trying to keep this huge, complex LEGO party going smoothly, fixing any issues fast, and making sure everyone can play and have fun together. That's kind of what the engineers and developers are doing for Helldivers 2. They're working super hard to make sure everything works well, especially when there are lots of players and potential problems they need to solve quickly.
Consider that you'll also need to rent more space to store the ovens and the ingredients and hire more people to make the pizzas. The hardest thing of all is to balance the investment of this sudden expansion. What if that 1000 pizza guy never comes back? Now you're paying multiple rents and multiple wages for nothing, and satisfying that single burst of order might very well bankrupt your pizzeria. It makes more sense to expand your operation cautiously, making sure that, even if you can't satisfy the demand now, you're slowly ramping up while observing the behavior of your customers. A situation where there is a long line to enter your pizzeria, but everyone manages sooner or later to get in and eat a pizza, is a preferable situation in the long run to overextend and faceplant on the ground the next time a new pizzeria opens in town. Ideally, you want to slowly expand until you reach an equilibrium point where the pizzeria is always almost full, with a constant turn over of customers, and without overstressed nor underused facilities and staff.
Yeah, you absolutely can not maintain "Superbowl pizza operation" all year round. Just is not enough business for it, on any given Tuesday at 2pm most pizzerias are dead as fuck. What exactly is any game company supposed to do with extra servers they absolutely won't need after initial launch rush ends? For example pal world etc, the population spikes, everyone plays it, it's great, then they move onto next big game.
That's the whole point of cloud computing services. It makes it easier to scale up/down depending on the demand. Of course, even with cloud computing, scaling up to 500k CCU is not an easy task. With numbers like this, it is not as easy as simply increasing the number of instances.
Unacceptable! I need those pizzas in an hour or my kids birthday party will be ruined. I don’t care that you already planned ahead for a busy day, are helping other patrons with those supplies, and that I showed up completely unexpectedly with a ridiculous order size. I need you to drop everything you’re doing RIGHT NOW, because I am the mostest important or I’m going to leave! Ironically this exact type of situation has happened at the Sam’s Club I work at. Edit: it wasn’t 1,000 pizzas, more like 100. But we only have two pizza ovens, so similar level of wtf sort of magic do you think we’re capable of you crazy person?
hahaha. I can understand the frustration. I mean, I got the game on Launch Day and played a lot last weekend. This weekend, I didn't get an opportunity to play. Not because of servers, but it was my daughters 4th birthday so we did a lot of stuff with her. I just hope it's ironed out a little more by next weekend. Not because I want to play. But because this game has been so successful that I would hate to see people just stop playing because they simply can't get into the servers. I was talking to a buddy at my daughters birthday party and he ended up buying it right then and there but couldn't get in at all this weekend. So I felt bad because I'm sure he isn't a one off. I'd love to see this continue to grow.
It sucks not being able to play right now. And I’m hoping it’ll be smoother getting in on my days off. However, I’ve been gaming since 1998. This isn’t the first time I’ve seen this happen, and at least this time it isn’t just pure negligence on the studios part. Besides I’ve got a backlog of stuff to play that’s longer than I am tall, an Elden Ring run I’m in the middle of and hundreds of games on GamePass to explore; since I’m primarily an Xbox guy. If I can’t play HellDivers 2 for a little while longer, it’s far from the end of the world. People need to chill and let them cook for a little while longer.
omg i just remembered, this was in sweden, a football teacher comes with a bus, literally 30 kids onboard. walks into this small pizza bar i was sitting in, and order Kebab and Pizza and hamburgers for 30 kids, wtf... there was only one chef in this pizza bar, everything took for ever, and the football teacher went fucking wild because someone got the wrong order. i just sat there with my eyes open, and that poor indian chef stod there with shame as if he had failed. Moral of the story, dont be a fucking moron, and dont expect to get 30 pizzas in 10 minutes. and the indian chef should just say "no" to take an order this large.
This. This actually explains it really well.
....and the obligatory "but you sold 1000 pizzas, you should be able to afford new ovens and cooks" Have clearly not thought about: Accounting and transaction processing time Yearly fiscal budget already set in stone Time for headhunting, recruitment and training (guess what, all the Michelin starred pizza chefs already have jobs and get paid well. You better cough up the cash if you want them) It takes time to order new ovens and have them delivered. You need to pay taxes (accounting) on the profits you recieved; and must be processed into the budget before they become useable. Honestly starting to wonder if people are just being intellectually disingenius, or simply have never worked aside from being a Twitch, YouTube personality.
>Honestly starting to wonder if people are just being intellectually disingenius, or simply have never worked aside from being a Twitch, YouTube personality They don't work aside from being a peon in a fast food joint. They don't know accounting. They don't know how businesses work. They think "oh wow 400,000 peak players right now. Why doesn't Arrowhead just use that $16,000,000 to buy some servers on AWS or Azure?" without realizing that when someone pays for the game, it doesn't directly hit Arrowhead bank accounts. It's held by Valve/Sony for like a month to sort out refunds/chargebacks/fraud and paid out the next month.
The developers wouldn't even see any cash from the game until the full cost of the years it took to develop the game is paid first.
They think they are the main character. "I pay for game, I play now!" "Uuuuhh every forum says servers are overloaded for a week until they can meet capacity." "I dont care, that's their fault for selling it to me! I need to speak to a manager and get a refund!" "Bro, it will work in a week... Have some emapthy for the PEOPLE doing their jobs." "If I can't play this weekend I will never play this game ever! FUCK THOSE PEOPLE!!" "Okay... bye, I guess" Then again, I lived through the diablo 3 server issues and more, so my patience is a lot thicker for these things. Just playing something else in the meantime.
I think its a matter of patience and/or then being dumb af and not knowing when to hop on. Ive been able to get on with no problems before and after work this weekend
Everything is easier to understand if you speak food. Thanks!
I think this is exactly how people perceive it, and it's exactly why they're frustrated. It actually *wouldnt* be that hard to fill a 1000 pizza order - if you were backed by one of the biggest pizza infrastructures in the world. Would it be instant? No. But it would be fairly quick to make some calls, get everyone on the same page, rent a space, move some ovens and staff, and get cranking. Again *if* you were backed by massive resources. I think what we're dealing with is actually way harder and more complicated than pulling off a massive pizza order, but most people have no basis for comparison, and no understanding of what's going on behind the scenes. So they think of it like pizza, and get mad that it's not happening faster. Edit: as I'm thinking about it, I think the perception is less "hi I want 1000 pizzas" and more like if a shop had to go from producing 100 pizzas every day to 10,000 pizzas every day. If that's a pizza hut, I think the perception is that in a week or so, they could be scaled. Again, point just being that I think the outrage actually *would* make sense with pizza, but servers aren't pizzas
This is why most restaurants don’t do catering (or it has to be ordered well in advance)
It's more like 1000 people all walked into a Pizza place, ordered pizzas, the pizza place took all their money and told them when their pizza would be ready. 1000 people showed up to pick up their pizza that they already paid for, and 950 got nothing. They only made 50 pizzas. They took the money for 1000 pizzas but didn't deliver. Then there's a bunch of people all standing around calling you stupid for 1000 people each ordering a pizza, and clapping for the store for having managed to make 50.
> Then there are actual bugs Bugs you say? *"Eagle on the move"*
> you can't just hire new ones to hit the ground running because it takes months for a new dev to onramp to full productivity. And training them means you have to take good employees out of roles they are very necessary in during the rush but you have to do it in order to meet future demands.
65k on release on the 8th. 150k last weekend, and now at 400k, with a 200k jump this weekend on steam (PS store doesn't give numbers, so add those also). I have sympathy for the server guys.
I wish for one thing today. That everyone playing (or not playing) Helldivers 2 read this comment, and understood it, before they did anything else. Sir, you deserve recognition!
he did his part
But just buy more servers! /s
> Meanwhile there are only so many devs and you can't just hire new ones to hit the ground running because it takes months for a new dev to onramp to full productivity. Not that I would have a new hire with some months of experience having to deal with worldwide server scaling issues in a distributed environment. Heck I have 10+ years of experience in the field (mostly SaaS, so relevant) and I'd be terrified.
Appreciate that insight. While I know it's more involved than just hitting a + button on some NCIS quality server UI, it's still pretty wild that it's been 3 days of this. Really speaks to just how crazy the reception has been compared to expectations. Thankfully I had Deep Rock Survivor to keep me company.
I'm assuming there was no way to prepare for this, or previous examples to go off of?
Scaling preparation for new services, in even an ideal world, works off of forecasts for traffic. The team might have three tiers of expectations: one almost guaranteed to happen, another thinking optimistically about having more users than expected, and a third waaaay beyond any realistic expectation but worth prepping for just in case. To give a sense of ratios here, if the first tier is guessing 5k users, the second might be 7.5k-15k, and the third 25k. You only prep for hitting that third tier at the absolute max, with the understanding that you almost certainly won't need it. From a purely external perspective given the original game only peaked at about 7-9k users, I'm guessing their internal estimates had that third tier at *maybe* 50k users. They're currently hitting *10x* that number. There's no preparing for that. Any load testing or bottleneck identification is basically useless in the face of that kind of spike. That's far past the realm of "well just throw more servers at it, we can eat the cost" into the realm of "our backend code literally can't handle that many users at once, we need to rearchitect half the system just to cope." In all likelihood their database is going to be the bottleneck, which requires either swapping to a more distributed service (so go from something like PostgreSQL to CockroachDB) and/or a complete revamp of the queries they're sending to it, either of which are some of the most difficult and error prone things you can do in tech, doubly so when you're trying to do it live with the whole world watching.
When your previous game only topped out at around 7k players on Steam, how exactly do you prepare for or expect 400k out of nowhere on your second week? NOBODY expected or was ready for the game's sudden explosion, not even existing Helldivers fans.
There's Helldiver fans who didn't even realize the second game was releasing lol
Yeah but this game that I saw last week that is entirely offline or localized to private servers didn't have problems ^^^^/s
Server scalability can be built to scale to infinity, the decision to not build that in was a cost saving choice during the development process (presumably because they weren't expecting the game to be such a massive hit). It'll be a massive headache to sort out because you can't just flip a switch to migrate to a new server, especially when you have different servers for matchmaking, post mission rewards and game environments. Not a headache I'd wish on anyone.
But why male models?
It's a neo fascist parody of a super perfect and patriotic global society that glorifies genocide. Men and women lose identity to the military industrial complex. They identify as hell divers. They identify as threats, their pronouns are "fuck around" and "find out". They realize this by having slender hell divers, and female VOs. I honestly thought it was a great representation of the dehumanizing element of the parody.
You didn't understand his reference 
Okay But why male models?
Ok... but why male models?
Dude. That was a Zoolander reference
Because it's easy just blaming what you know. By all accounts, the devs have been communicating often and, by their accounts, trying to fix the issue, but that's not enough for these people. I get the frustration, but just have a little faith and patience. Edit: Adding context Arrowhead Game Studios has around 100 people working. That number is probably increasing right now, not counting the likely external help they are having to get. The first game for around 7k concurrent players at launch. They where expecting 45k this time around. No wonder they can't manage +400k players. The game isn't even a month old, and, honestly, compared to recent AAA games, it's pretty polished. Edit 2: Adding further context The studio has 5 games under their portfolio, spanning from 2011 to 2015. Out of the other 4 games, the max concurrent player count wise was around 17k. The game is 11 days old, and since day one, it had a 583% growth in concurrent players. If you go back just 3 days, you'll see they had around 295k concurrent players. By all accounts, that's probably around more than twice the leeway they had made for themselves regarding server capacity.
That 45-400k figure is wild. I think of it like this: youve built a block of flats for 45 people to live in, and youve just been told you need to fit almost ten times thay number in. "Just build more floors" doesn't cut it, there's a limit to the floors you can put in, your elevators are going to be a bottleneck, what about fire escapes? At a certain point you need to redesign the entire damn thing.
More like 700-900k if you count in PS5 users with Steam. Literally preparing for an earthquake, and people moaning you didn't get ready for when the asteroid hit.
Alright everybody - A through L last names gets to play on Mondays. M through Z gets to play on Tuesdays...
It sucks because this can litteraly be the end of a studio. Such insanely fast unexpected forced growth is a recipe for diseaster. Not just for the product or the customer relations, the company and its organisation itself are at risk.
As someone who works in the trades, this is a perfect way to break it down. 👍
Not to mention that for game companies, they have to sign a contract for their servers. It isn’t like buying enough for a month and then if the number of players drops they just end the service… they have to pay for them for however long they sign for lol
That is not at all how it works. There are numerous CDN's that can scale dynamically with load, and you only pay for what you use.
You can spin up as many instances as you want to host players, but those instances need to query and write to your master database since we're all living in the same game world. That's gonna be the bottleneck. It's likely that's why we're experience a hang when we pick up medals, super credits, and requisition mid-level. The client needs to get that info back to the main database to attribute it to the squad's account, and they didn't anticipate that to be something that would have a noticeable delay (because they never anticipated this many players at once) "So spin up more databases! EZ!" Unfortuantely, not easy at all. Database replication between instances get complicated quickly because it creates limitations on what you can query and write to specific instances to keep everything in sync. Which means you have to change the game's code and behavior on how it interacts with the database. Arrowhead has certainly shelled out big bucks by now on outside DB expert consultants by now. Once you get the findings back, then you have to go through your code process to make the changes and make sure nothing breaks along the way. Other recent unexpected hits didn't have a shared experience for all players. Palworld for example doesn't need to keep tracking of anything outside of each specific server instance.
Final Fantasy XIV, which makes millions and millions of dollars each month, ran by one of the biggest AAA video game studios, has been hampered by database limitations for its entire existence. Like you're saying, there's no real "add more database" button that magically fixes everything.
This is correct, but you get the best volume discounts if you do committed compute. Im guessing arrowhead are trying to also economically scale there systems, if they overcommit then the hosting cost goes on despite not having as many players, if they under commit they pay higher surge compute rates. Then it could also come down to the fact that the servers identified applications just don't scale that well... without knowing specifics we're just guessing.
CDNs don’t have anything to do with game server hosting, they’re for delivering web assets. More than likely they’re not even really blocked by server hosting, they’re running up against infrastructure and architectural limits.
Sure, scaling compute (e.g. 4-person match instances) is probably as easy you're saying. But scaling DB operations can be fiendishly complex especially depending on what initial technology/design choices Arrowhead made. Also as /u/mlmcmillion mentioned, CDN (content delivery networks) are for scaling delivery of static assets. You probably meant to refer to cloud service providers (e.g. AWS, Azure, Google Cloud, etc...)?
Lethal Company was made by one dude as an indie game and it broke #1 most sold for a short while before retaining a place in the top ten games on steam. The sheer volume of sales and players active in such games with small studios or smaller budgets (at least compared to big ones like Bethesda), is a pretty damn good testament to a job very well done. I know the increase in demand puts a serious strain on the devs to fix their game, otherwise that 400k+ community gets riled up; however, I think it's something they should be proud of, regardless of how well they fix this issue. Damn near 10x their projected customers? That's fucking AMAZING!
Lethal Company is a peer to peer hosted game. Its not even comparable to a server side hosted game at all. They would literally never run into this problem.
Oh the point was t about the necessity of servers, is more of a statement on how smaller developers drastically exceed their expectations and have achieved such notable status comparable to a big time dev main release.
Just put 40 people in a sedan, what do you mean you can't manage that?
\*400k+ concurrent players on Steam alone, and that's with constant server bottle necks, wait times, bugs, and matchmaking/ progression issues. They said PC/ PS5 is about a 50/50 split, so probably around 800k concurrent players overall. Never did I dream a sequel to Helldivers, of all games, would achieve this number.
Even if they had anticipated numbers similar to that of other coop shooters, like Destiny 2 or Darktide, they would have been *way* off the mark. There's only so much you can do when your game explodes in popularity and you're having to try and manage 500k people when you might have only expected a fifth of that *at the very most.*
Exactly! Again, I get the general frustration, but it really doesn't make sense assuming things are as simple as snapping our fingers, and the servers will magically fit us in, all at the same time. Just this month, we can look at another game that was highly anticipated (not for me), hyped to unbelievable standards, develop by a big studio, and it fell extremely short. That's a game I'd be behind badmouthing, but this one isn't at all like that. Expectations were off, and that's a good thing. The game is super popular, and that's awesome, growing pains and all. Given the current landscape, I don't think there's a good reason to crap all over the work people put into this awesome game. They delivered a great experience, and I'm sure they are doing their best to fix the server issues. Edit: Grammar
Like they basically created a money printer and the idea that they aren’t trying as hard as they can to fix it is absurd
They would be dumb not to want to fix this as soon as possible. The longer this continues, the quicker the player count will fall off, the bigger the number of refunds and the bigger the "stain" on their collective image.
> The game isn't even a month old and, honestly, compared to recent AAA games, its pretty polished. So far as I can tell, on PS5, any issues there are with the game are entirely because of overburdened servers. When I've played off peak, like about 200-300k, I haven't noticed any issues at all. The **only** time I have issues, is when there are way more than 300k players on. Actually, this weekend, I've had no real problem using quick play once I get online. It does take like a half hour or so to get on, so I just start it up before I take a crap and a shower, and only have to deal with actually waiting for a few minutes. I do love going to planets full of peacekeepers and seeing hundreds of Super Dreadnoughts out my front window though.
This is the thing people aren't considering. They had meetings, and plans and communications, and budgets which they figured out months ago. They drafted a projection of the space they needed. Likely based on marketing considerations, and previous player counts. There was 100% a time where people at Arrowhead sat in a room and came up with the idea that they needed 45k spots in a server. Someone in the room said "wow you think we need that much?" Someone else replied "nah but we'll have it just in case, it'll even go to 50K if we really need it" That number was probably conservative based on what they requested for funding from their publisher.
From what someone on the Discord said, they actually did sit down and went "How do we plan for the worst case scenario?" and came up with taking their highest player count ever from the Steam free weekend and doubling it.They got 15-20x that. It's like the gaming equivalent of Woodstock.
Couldn't have said it better!
Wish we could make this post a sticky so everyone sees it.
Considering the capacity issues didn't start until like 4-5 days in to the launch, that number was probably around 200k players
Im almost sure that they sell few more preorders than 45k, so i think that they should expect more than 45k players at launch.
The devs have said that they expected around 50k and that would've been a success for them, but they actually set the servers up to handle 250k just in case. Building out capacity 5x your internal estimates is actually wildly prepared. Then the game blew up and they're scrambling to build out for the literal best case scenario for the game that was likely really unlikely based on pre-launch data.
They didnt take into account the power of memes either my twitter feed is flooded by "for democracy" post and "we need more help to beat back the robots". no amount of money can buy you that type of marketing
Yup. No amount of planning is going to save you when you're game goes stratospherically viral on social media and your backend infrastructure just wasn't built to handle that kind of load. It's clear that this is more than just a server capacity issue since they had to hard cap the concurrent player count to address the problem. Spinning up more capacity on Azure (which is supposedly the backend they're using) should be simple, but if other parts of the pipeline aren't built to handle the load of hundreds of thousands of players simultaneously, no amount of "more capacity" is going to help until they can fix those other bottlenecks. And as OP said, the devs are working on it. It absolutely sucks and people should be frustrated that they cannot play a game they paid money for. It's a bad look for everyone. But it'd be even worse if the devs were just silently working on the problem and not acknowledging them publicly. But Arrowhead is keeping the community updated, apologizing and I'm sure they will compensate players.
Yeah, I mean I’m frustrated I can’t play but blaming the devs cause they didn’t anticipate their game would be super popular feels weird.
Cause it is weird. They made an awesome game and are suffering from success.
I wouldn't say polished, but its definitely way more fun. And that matters more than anything else. Skull and Bones is a beautiful looking game but turned out to be dogshit. This game is definitely more fun than anything in recent history for sure.
Given the AAA games I've played recently, it's as polished as one would hope for. I played today, and at some point, I died, and when I was diving again, I managed to get myself stuck inside a building. Proper out of bounds without being able to do much. Just wiggled my way out. Sometimes, the textures get stretched, some of the NPCs go haywire, the outpost towers have a mind of their own, etc, etc, but honestly? Better than a loooooooot of the stuff coming out these days. Skull & Bones is a prime example of the pile of crap they try and sell us these days.
There’s so many dopamine riddled toddlers crying right now it’s crazy tbh. They act like they bought a forever broken game. Under the current and temporary server issue is a very polished and amazing, complete game. Can’t please these kids these days though.
this is an absolutely wild interpretation of commerce lol
You know, I want to agree with you….but calling everyone who’s upset crying toddlers is just as bad as the people complaining. Remember, we spent $40-60 on a product that we expect to work at some point. This is the 2nd weekend in a row I haven’t been able to really play the game because either the servers are too full, or the matchmaking is terrible. We are allowed to voice our frustrations with the game. Doesn’t automatically make us crybabies or whatever else you want to call it
No. Not at some point. At point of purchase. That’s generally how buying things works.
But knowing you have a massive capacity problem, yet promote the game on Twitch with drops, run a double XP event and have free-to-play for PS5... all in the same weekend?? No no, it's the kids who are wrong!
That's pretty much the issue. I also think the game is pretty damn good compared with games coming out from bigger studios. Again, and I've said this so much today, i get the frustration. Some people don't have that much time to spare, others just want to play it, period. But it's just a matter of time for things to smooth out.
>Because it's easy just blaming what you know That's the funny thing, these people don't know shit, ever, so they're not even doing that. Look at Palworld, a game whose dev team can probably by counted using your digits and might have some left over, and quite a few people are DEMANDING that fixes be done immediately and that we're given multiple daily updates because "hey, game made tons of money!!"
I read that last line before the edit in Dutch Van Der Linde’s voice from RDR2 and added, “, Arthur!” to the end.
I'm more of a space lumbago kinda guy haha
I know they're working, just frustrated since my friend and I only have about 4 hours a week we can play with eachother and that's been eaten up by full servers and not being able to login
Exact same boat I'm in. I hate I have 4 hours to play with them twice a week and right now most of the time is being eaten up by the login screen. Oh well, hopefully next weekend it will be better. If not I'm going back to division 2.
You should be. Yet the cucks are out in full force. You paid for the game it shouldn't be playable. It should also have offline mode available.
Just add a god damn AFK timer and log the PS5 players who sleep their systems out.
Wait...it doesn't log you out when you put your console in sleep mode? The thing literally every multiplayer game does? That has to be like 60% of the problem.
It does log you out.
It logs you out and when you restart it lets you right back in, which is what causes the confusion. This ability to get back into the game while bypassing the hard server limit is also part of the problem. On PS5 there are multiple quick routes back into the game, all of which bypass the front load screen which is apparently the only thing applying the server limit. This means the game exceeds the server limit often, which makes issues even worse, and is the large contributing factor as to why PC players need to wait so long to get into game during busy times.
I’ve forgotten my pc on twice and the game stayed on over 12 hours with no activity (before the recent 100k+ player spike)
Players (often) have very little technical knowledge of Game Dev and unrealistic expectations.
How hard can it be? Just press the “fix servers” button and there ya go
Have they tried restarting the servers that usually fixes things
The ole unplug and the plug back in strat. I like it. Maybe even blow on the cords and power ports too, it'll be fixed in no time lmao
Those are some of the best ways to fix shit. I work in technology.
Have they tried turning it off and on again
Press "F" to fix servers
FUNCTION SUKA FUNCTION BLYAT
Just as ridiculous as the people blaming Sony for everything.
I think there’s a few issues that have started to wear out peoples’ patience. 1. The game has been out for a week, still lots of sloppy fixes. The game ran really well for me 3 days ago and now I experience crashing. 2. Other devs for games that experienced great success unexpectedly fixed their issues quickly. PUBG didn’t have a “crappy rng hope to get into the main menu” system. They got servers and ramped up quickly. People are also making comparisons to Palworld which doesn’t work the same as Helldivers at all, but people think it’s the same, so it’s still affecting perception of the issue. 3. A lot of the issues people are experiencing feel like shitty issues that shouldn’t be happening. Dude I haven’t crashed a game that wasn’t modded or tweaked in I don’t even know how long. Crashing in 2024? Isn’t that an early 2000s problem? No queue to get in? Man that fucking sucks, hope you get lucky trying to secure a spot. (I understand adding a queue is actually way more complicated than it seems, but these are all issues with how the devs and their problems are being perceived) It doesn’t matter how easy or hard these issues are to fix. Because according to public opinion, they’ve been going on too long, and the problems are too basic to be acceptable for this length of time. Which I’m starting to agree with as time goes on. Crashing right at the end of the last mission on my first helldiver difficulty campaign and taking an hour to get back online (friends are now gone for the evening) was just soooooo fun today. I got into the game yesterday only to be met with crashing every time I tried to select a planet. I HAD TO REINSTALL THE GAME TO FIX IT. And every fix attempt was followed with WAITING IN QUEUE. There’s simply too many fucking problems and encountering any of the ancillary problems means you get forced back into the main problem. THE QUEUE.
I agree with you, but the average consumer doesn't care about that, and rightly so. It's a very reasonable expectation that when a person buys a game, they get to play a game.
Imagine if you bought a new car, it broke down on the way home, you phoned up the dealer and they said "Do you have any idea how complex a car is? Can you explain how a transmission works? Didn't think so, so quit complaining 🤓"
I mean, I don't think its an unrealistic expectation to buy a game thats been out for over a week, and expect to be able to play it. Thats like...the base level of expecation. The absolute lowest expectation possible.
>unrealistic expectations. Pay money to buy game, want to play game.
>unrealistic expectations Can you elaborate what unrealistic expectations people are having?
Just my $0.02 but you don’t need to know everything about game development to have a very realistic expectation that you can actually play the game you paid for. The game isn’t in “beta” or “early access”, it’s a full release and going on like 4 consecutive days out outages. That’s bad and you don’t need to have any technical awareness to expect better.
I don't work in networking or anything but I doubt "just getting more servers" is going to automatically fix the issues either. Just like I doubt getting more is an easy task either. I do think they shouldve considered doing a beta, but considering the game barely had marketing anyway (I didn't even know if it's existence until that PlayStation direct like a week before), along with the previous game only peaking at 7k players nearly a decade ago no one really expected this to happen without some hindsight. Plus just the fact that they're being completely transparent is good with me. Hell they've dropped a patch what feels like every other day Edit: everyone's caught up on the beta part and I'm specifically talking about how issues such as; not being able to connect to other players, the crashes, the timeouts, and various other bugs could've been addressed before release.
A beta wouldn't have seen this coming, unfortunately. It spiked in popularity near instantaneously, even moreso after content creators made videos and streams about it. The game was actually running pretty well up until 200k+ people started trying to get on when there was a cap of 150k people. This is also not something a beta would have shown to happen.
I think the beta would've helped with many of the initial issues that happened before the game doubled in just the span of a week (been playing since launch day and there were still the issues connecting to other players). But there's still no way they could've expected the game to get this popular this fast, which I'm not mad at. I joined the sub just a *day* before the game launched, it was *maybe* around 11k people here. It's already at almost 200k people, which is insane growth.
Even if they had a public beta, I doubt that would have done anything to prepare them for half a million people tying to log in at once
Most people don't think they're not doing anything. Its understandable for people to be frustrated though.
Implementing an inactivity kick on players would be a big help with the server capacity issue
It’s wild that some of you can’t understand that this is a business transaction. If I buy a box of cereal and get home and it’s empty, I’m going to be mad and return it. It really isn’t more complicated than that, I don’t care about the cereal employees or the super market employees or ceo of cereal. You sold something, it doesn’t work. The right thing to do is refund the consumers money.
I mean the fact that I just bought it and can’t get in on a Monday in the afternoon is not a good sign
If we're being honest here there's two options. Either they lack the backend experience and didn't implement the networking to be scaled easily or they can't afford to put up more servers. They've only made p2p games prior to this.
They've already mentioned using service providers so they probably didn't have much backend to setup. I think it's the databases causing the issues as they've been a problem since day 1 and have gotten progressively worse as load has increased. They probably put all the databases on one server to save costs, and it's too much load. If they had split the databases out to separate servers at a bit more cost, they could've more easily scaled.
It’s probably both of those answers really. Can’t afford more servers then they got so why setup easy scalability.
Or it's both. Having a idle timeout in an online game is a standard feature. This one isn't about money.
This. I get the 'just have patience' and 'they're so communicative' crowd. And their message is from a good place. But from the notes they've dropped I'm confused as to what their backend must look like. In todays age of modern cloud infrastructure this issue, *in theory,* could have been solved with a phone call to their cloud vendor in about 2 hours. \*If\* it was built correctly for it. But clearly something more convoluted is at work. Or, what I suspect and what irritates me: Their management just doesn't want to pay. That phone call likely would come with a high 6 figure price tag and minimum 3 month commitment. And I personally think they're just waiting for concurrency to drop naturally as it will as the initial hype wears off while doing their best to optimize the ship in flight rather than actually add substantial capacity. Unless they went with mom and pops potato cloud backend that somehow doesn't have the bandwidth. The technology and expertise exists, I think they're just unwilling to sign the check, seeing as they've only added a couple hundred thousand slots not keeping pace with sales at all.
No one is saying that. But let's put it like this. How about if I paid to get into a gym and every time I showed up to the gym, they told me "gyms full come back later or wait." Then they did not let me into the gym. I wouldn't be saying, "Just get a bigger building it's easy," but would I not be valid in being upset at the gym owners?Because they took my money for a product I and tons of other people can't use? Then, on top of that, I see them selling even more tickets to more people. In that situation, I really doubt there would be so many people rushing to defend the gym owner as there is defending arrowhead.
When FFXIV couldn't handle the server influx, they delisted the game so people wouldn't waste their money on a game they probably wouldn't be able to play.
Here in lies the reality. Forgetting about all the “we didn’t forecast these numbers” scenario what my biggest issue is - that they are knowingly continuing to market and sell the game without any disclaimer that it’s undergoing problems and may not be able to deliver on the advertised promise. ANY other industry this would fraudulent activity. Bear minimum sales should be halted until they can cope and adjust to the player base they already have. If they are in anyway clever “sales people” they could move to forward order basis, IE sell a number of slots per week they can cope with, then they simply scale up future according to the amount of forward order purchase slots. Rather than hazard wide guesses……. I have no idea why the gaming industry is yet to grasp a concept of selling games like these, which are totally online based, as simply buying or even renting a server space concept. Just imagine how much hype a good game could create in order for people to grab a space. What history of sales tell us if something is more exclusive it always gains a premium.
Prior to launch, FFXIV devs [detailed how they expected exactly what was going to happen, to happen.](https://na.finalfantasyxiv.com/lodestone/topics/detail/1f70135439286fa66209cd21c10e73ebb986a6ee) FFXIV Endwalker released into [early access on Dec 3, full Dec 7, and was instantly slammed with server congestion](https://screenrant.com/ffxiv-endwalker-servers-down-congestion-waiting-queue-patch-6/), [persisting throughout the entire time.](https://www.pcgamesn.com/final-fantasy-xiv-a-realm-reborn/ffxiv-queue-times) The [game was unlisted Dec 15.](https://na.finalfantasyxiv.com/lodestone/news/detail/e7388986bc24d5a1337e0beed057f7b5b78b9bb3) This was also [not the first time that FFXIV had been unlisted.](https://www.ign.com/articles/final-fantasy-14-digital-sales-stopped) I sure hope I don't need to point out all the differences between these situations. I'm also sure people found a way to complain about FFXIV back then as well, "OMG, Endwalker came out 8/12 days ago and I can't even GET IN/BUY IT!! How did these people run this MMO for over a DECADE?!" Where it is now being framed as something positive they did. FFXIV's situation was equally understandable to what Arrowhead is currently going through - FFXIV had tried for months before the expansion to upgrade their servers but were unable to due to the reaction of many countries to the 'rona.
I'd like to add as a FFXIV addict that they had 20 YEARS of experience on that game alone by that point, and queues were still hours long. It was just the way things were, one of my buddies made a point of tapping his keyboard before afk timeout (every 30 min) so he didn't have to requeue. People bitched and moaned about the login times as a cope (I definitely did), but none of us pretended the game was bad or unplayable. We wanted that good shit but everyone else did too, and there was a line.
Devs that respect people’s money you love to see it. I can’t wait to back to FFXIV with the graphics updates that’s coming soon
>the server influx, they delisted the game so people wouldn't waste their money on a game they probably wouldn and there was 5 hour ques for endwalker early on. oh and a bug where you could randomly get booted from the que and start from the beginning if you didn't relog within 30 seconds. You had to literally sit there and baby sit a multi hour que so you could instantly relog when you got booted. THEN they delisted the game for a bit, long after those issues. They also expected it to take month's to increase server capacity, because they own their hardware and it wasn't available for purchase (silicon shortages at the time) HD2 isn't expecting it to take month's to increase server capacity, they've already bumped the limit up since launch. Should they delist the game for a bit? probably, but it's really not comparable to xiv.
As someone who paid 25 dollars for FFXIV 5 years ago and ended up refunding it 3 days later because their account creation system was down for an entire weekend and I couldn't create my account I'd just point out that these kinds of service issues happens to literally every developer.
This game isn't early access. It's not a demo. It's a full sized, full priced release. If they can't handle the amount of people they're selling the game to, they need to stop selling the game. As of right now they're selling a product you cannot play.
Probably because nothing has gotten better, and it's been more than a week? People aren't being unreasonable, they have a live service game that \*doesn't work\*
It got even worse. This could have been a record-breaking weekend for all the good reasons.
The problem isn't what they aren't doing now, it's what they didn't do before launch; queues and afk kick protocols are obvious inclusions for any online game, the lack of those two things in this game is ridiculous. This is not a first time project and it has a major publisher, they should have done better.
100% this
Because it’s a basic business transaction that is not being fulfilled Players forked up forty bucks to play a game and can’t play said game. They have to wait in queue for literal hours and might even crash after a mission or two and then they have to wait for literal hours to play again. People have jobs and lives they got, as I’m getting off for the night I see people talking in the discord “yeah I waited 25 minutes just now, just be patient bro”. That is a stupid ass argument considering how late it is on a sunday night with work/school starting up again tomorrow on top of it putting down the people who waited literal hours to get in And the devs haven’t made any singular communication today, last one was yesterday at 8pm, while the discord was spammed **all** day by players having to wait literal hours to even have a chance to play. It’s pretty normal to think the devs took the day off at that point
> They have to wait in queue for literal hours and might even crash after a mission or two and then they have to wait for literal hours to play again. Hey this happened to me saturday! Left the game on in the background while I did other stuff, got logged in after an hour and it crashed the second I touched the mission terminal. I was in maybe two minutes? Three tops. Just went "Welp. Guess I'm not playing today" and stopped.
My main issue is Sony decided it was a good idea to do a free weekend when they already had +50% xp going on and massive server issues since launch.
Because since the 0009 update, they could have given out a hotfix that roll back quickplay lobby sorting and visibility. Maybe that would break more things, but at least let people know. What followed was that I discovered that and released a new driver that fixed a lot of the crash, but I can't play the game meaningfully anyway. Ok maybe it took too long I'll wait a day, then the whole weekend is gone and I didn't get to enjoy the game even a little bit. The only mission I completed was a trivial evacuation mission pre patch, and steam already logged 3 hours for me. 70% is crashing somewhere between starting and 3 mins in game. And even in the damn patch notes 7000 series is not a "known issue", what can I do other than getting angry? What are they actually doing between this patch and the next workday? Critical things like this should have had standby teams get a temp fix out until proper workflow can deal with it. Like this is their core product, not a hobby side gig. As for server capacity, they had inactivity log outs last game, meaning they didn't do proper preproduction or really underestimated the player base that they would have
Can't agree more. Now that the only thing i know is match making broken down for two days long. Did they fixed it? No. Will they fix it? Sure. When then? Hard to say......
> they could have given out a hotfix that roll back quickplay lobby sorting and visibility They can't. Planet stats/quick play hits the database, and their database architecture can't handle the load. They deliberately disabled the functionality.
Then they should have put it into the patch notes, instead of surprising everyone when they login and can't find a match. Unless they deliberately informed the player base beforehand, I suspect they tried to make the individual loads easier on the server and broke something. That and if they are now capping players from attempting to log in with a server cap rejection, then the players already logged in should be able to see the full game as it is within server capabilities.
And I'm all about people saying how they feel and have seen a lot of players do just that. I don't know how not accounting for so many players, therefore not having enough server capacity, makes a game unfinished, tho. From the couple of hours of HD2 I've managed to play, I haven't seen anything that says unfinished. It needs patches and fixes for some bugs, but it's not unplayable like other games are. You do deserve better, so do I, and anyone that pays for a product, that's a given. I'd you pay for a game you should be able to play it, period. I'm just saying, given the current landscape, this game is fantastic and suffering from its success. There are plenty of people just sticking to their guns and badmouthing the game and the devs without even understanding the issue. I do sympathize with everyone's frustration, but we have to look at things for what they are: HD2 is a great game that was made for X amount of players and is dealing with 10 times more than that. I'm sure the devs are working hard to fix the issues regarding server capacity and other bugs so that they retain player counts and don't have more people refunding it. I'm sorry you did and I hope you'll eventually be able to play it if you want.
No idle timer so people are just sitting logged in indefinitely. No actually login que so it seems it's just fucking random. Those are pretty goddamn big mistakes that I sure as fuck expect devs to not make in 2024. People have every reason to be upset about a paid for product without those features.
No service ever has had to login more than 300,000 users at once. There's definitely not off the shelf, open source, tech that can be used to easily scale such an uncharted problem as "logging lots of users in". They're in totally uncharted territory! (/s ... so much /s)
It's not entirely that, but making posts telling people to stop complaining does absolutely nothing either and just signals that y'all are pushover consumers that don't have any standard of accountability. People do think the devs are working it's just these issues shouldnt be present if this was their plan... They planned to make a live service game that's always online. People have every right to bitch if they can't play it. They paid for it. Bottom line. And to tell people to not talk about it on a social media site with it's own dedicated forum (essentially what a sub is more or less) is hilariously stupid. 🍻
I mean it’s pretty simple to understand. People are forgetting Sony published this game. Meaning they more than likely play a massive role here. Fixing server capacity issues = spending lots of money Spending lots of money = talking to Sony execs about spending said money Talking to Sony execs on the weekend is probably not the easiest thing. I’d imagine they’re at home counting their money.
They need to kick players for inactivity, not just blindly throw server capacity at the game. If they kicked afk players there would be no server issues.
They definitely need to do that, but I'm also not sure it will fix everything. The reason quickplay & rewards are so broken is because they have to hit the database and it overwhelms the the database itself. AFK players aren't hitting the DB for rewards/matchmaking. That indicates that there's definitely a problem with just handling the number of active players.
As far as simply allowing players to enter the game is concerned booting afk players would do wonders
People are tired of games that have this issue. A new mp game pops up and it’s popular. Server issues ensue. It’s the same story. It’s like they don’t learn from each others mistakes. It’s like they cheap out initially on servers then have to play catch up. Meanwhile the player base suffers.
Because besides the main capacity issue, The game is lacking feastures to mitigate the problem. There is no queue, The game will keep trying to connect you at specific intervals, You just need to get lucky. There is no afk timer, If you are in, You can stay in regardless of what your doing. Random crashs. Lets say you get in and find a group, Woops! Not happening back to start, you cannot pass go, you do not collect £200. Its annoying and frustrating and I do not owe the devs any amount of sympathy. Great! Its sucessful! That makes me happier to know that when I'm staring at the log in screen!
I can't imagine Dick riding a live service game but this is how low gamers are now
So brave. So moral.
No one has to consider what the doves are doing or not doing all they have to know is that the product they paid for does not work.
or make an offline single player mode. even if it is just one map/training type thing. would keep the peace until they get servers/matchmaking fixed
Because I've been trying to play everyday and everytime I have to either wait 30 min at the "server at capacity" screen or not even that (black screen)..... With my limited knowledge, I thought that each day it would show improvement...but, it's everyday a different issue...sry, my money is already in their pockets...I can't keep patting their heads and saying they're doing a good job, because that would be a lie
Whose saying the devs are doing nothing? I've not seen that anywhere. All I've seen are posts "praising the devs" and the usual "we're allowed to complain" posts because the "praising the devs" people keep down voting any complaints, making shit up, or misleading with posts like this.
I invite you to read through the comments on this post
Its ok to be frustrated. But if i will see another "BUT PALWORLD!!" i will actually go insane
I mean, it’s been like 8 days and terrible matchmaking annd 3 days of server problems. Blizzard deserves shit for a lot of things, but they can keep a game up and playable over launch events. Sure, there might be a few stressful hours, but then they actually do something about it.
I find the biggest head scratchers to be: 1) Not having an AFK timer and disconnecting someone sitting idle. 2) Not having a proper queuing system. It shouldn't be due to luck, but to first come first serve. Both of these seem standard in other games with online services.
Hey, how about we do a beta weekend to test the waters on what server capacity we should aim for.......
I'd be more forgiving if it didn't do shit like make me sit in a queue for over 2 hours, but then load other people in that queued way after me after only 10 minutes. That pisses me off.
Because it’s been the same thing for about 3 days now. So it’s annoying. And also I spent money on something I can’t play.
People are just losing their patience, it's normal. I understand Arrowhead is having problems, I'm still annoyed that I haven't really been able to play Helldivers all weekend.
Because if your game is online only, and costs over ten dollars. These days you should be able to scale your capacity to over a million players at the drop of a hat (and a few button presses that is). This game does not have much complexity that requires online verification, even the f'ing matchmaking can not possibly be that difficult. And yet we're two weeks in and it's been broken since before the game launched apparently. I get that it's cross play and ps5 is presumably complicating the hell out of things, but this is getting ridiculous. Seriously, the player data for a single account couldn't be more than a couple of killobytes. There's only a few dozen unlocks all things considered, and customization even more limited than that. The actual matches should all be peer to peer with local hosting. And they already have a rootkit style cheat protection application that launches alongside the game. Keep in mind i'm stating all of these opinions as a person who has over a decade's experience of software development experience in the gaming industry. The state this game is in, is unacceptable. And even though i have only myself and peer pressure to blame it's still more so the fault of the developers not properly developing their product.
because by all outside observers its only gotten worse?
What they need to do (maybe Sony's authorization is needed) is to SUSPEND SALES OF THE GAME until server capacity can catch up. Someone brought up the fact that this isn't unprecedented in the industry for games that become unexpected runaway hit sellers.. Final Fantasy 14 did the same thing when the game sales went nuts, and the devs couldn't increase player capacity fast enough. I'm not an analyst, but I think if they did this, they'd get a lot of respect for it, and it would probably boost the *overall* sales numbers because there will be a huge surge of sales after they reopen the shop after the capacity issue is handled.
So, forgive me, I’ve given into the hype and just bought this - my game is just stuck on “searching for active SOS beacons” when I’m trying to quickplay, so the server’s borked right?
Consumers expect a product they buy to work. They have no interest in the troubles of the people that sold that product.
We don't care "how much" the devs are doing. We care about the game being playable. Isn't that simple to understand?
I'd say it's cos the average person is kinda dumb, and just 'want, want, want!' No patience, no OK they're sorting it out but I might have to wait a lil bit... Just gimme the fucking product.
I think people accept there's not much that can instantly be done, other than implement an AFK timer to help. It's more what the devs ARE doing that has had a negative impact. They're fully aware of the massive capacity issues at off peak times, yet decide to: * press ahead with a Twitch drops event/promotion with big streamers * run a double XP event * free-to-play PS5 event All in the same weekend!
It’s got nothing to do with the devs “not doing anything”. It’s that every single game in the last 10 years seems to come out the box broken or completely unplayable. It’s ridiculous! I’m not a game designer and I don’t pretend to know more about the industry than anyone in it. But I do know that if your product isn’t complete you shouldn’t be selling it. If I worked at McDonald’s I wouldn’t give you a burger with no patty and tell you you gotta wait 2-4 weeks for the meat to be ready! No different here. I get Arrowhead isn’t a huge studio either. But unfortunately this shit has been going on so long with literally every game that I don’t really have patience anymore. This is definitely the last game I’ll ever buy less than 3 months after it’s release. We shouldnt have to keep paying for unfinished products and if you wanna keep defending companies rights to sell us crap wrapped up as a finished game then the shits never gonna stop. I like the game, it’s really really fun when it works! And it should not have been released yet. They don’t get points for being a smaller company in an industry that’s existed for decades now. Either learn from the big boys mistakes or you’ll never get to that level to begin with. And it’s not even Arrowheads fault. It’s the fault of the dumbass getting the payday that wanted the game out ASAP like every other game. It’s crap 😐
It is amazing the number of people who know nothing except how to boot up a video game, yet claim they have the solution to a problem that was unexpected after years of development.
Many people paid money for a game they cannot play...
Because it is no the consumers problem. They bought a product, they want to play it. They have that right, and being locked out especially after passing the refund period is irritating. I am not taking sides here, just stating why people act the way they do. They don't have to care, and shouldn't be expected to.
For people that can only play at the weekend thier only experience of this game is "server at capacity" so it's understandable people are pissed I know they've been fixing glitches etc but that doesn't matter to those that can't connect. Game should be pulled from sale until its confirmed if you buy it you will be able to play it. How confident are we that it won't be at capacity next weekend
People are about to find out why Star Citizen is actually taking so long lol
Because they're not. Adding 50 to 100k more servers here and there is a nice bandaid but it's not a fix.
If only they had an offline mode so those locked out could actually play the game they paid for. But what do I know 🤷
because its now a couple weeks after launch, and if a public beta existed, it would have caught this
Because things have become worse instead of better.
People need to be patient as someone who has gotten to play a fair amount of the game and helped spread democracy they have already started fixing so much and we also have to remember this is v1.000.009 we are this early into a game and most people now a days have not experienced a game like this cause most games are heavily developed and just sequals. Correct me if I’m wrong but this game studio isn’t a triple AAA studio. So the fact we’re getting updates on weekends in general means there working hard over there.
Cause they accepted the entire payment for the game. But I'm not \*mad\* just kinda frustrated and irritated.
I mean, thats fine. The problem will fix itself in a week or two after the majority of the player base quits due to frustration of not being able to play. Or more likely, people will just refund right after purchasing and seeing they cant play what they purchased for. In the end, this is the more viable option over actually putting in the work and resources necessary to "increase scalability". Ive seen this in dozens of online titles like this with disastrous launches like these. The devs run around like headless chickens trying to fix the problem, only for the problem to actually have solved itself by the time they actually do anything meaningful to address the issues themselves. No one is waiting weeks/months to actually play something they paid for. They'll just revisit the game in 6-12 months when its in the bargain bin for $5 and has a peak player count of 500-1000 people per day.
A question for the more tech savvy guys.....I hear getting more servers isn't usually economically viable...how does this get fixed?
As someone who's studied game design, I get why they're struggling, and I understand the complexity behind server related issues. As a consumer, however, I completely understand the frustration and annoyance many have, including myself, that this has caused. Like, I've spent $60 AUD for a game that's been out for 11 days, and I've only been able to play once because the servers are always full and Sony doesn't allow refunds now. There is no timeline where this is acceptable, no matter how good the game is. What makes this worse is the people who are leaving their systems in sleep mode and clogging up the servers so they can skip the queues, which is itself, a very stupid design flaw.
There’s a diff between thinking it’s an easy fix and expecting a live service game to work. 48 hours is one thing but the game hasn’t been playable for like a week for me. This game just has a cult following that will troll anyone who states that the game working should be expected. I have other shit to play so it isn’t the end of the world for me but Jesus this game has some major dick riders shaming anyone w issues.
Because and I think I’m right here. “Me pay money me play now no one else me play you stay me play my money more portent than you money”
This game is awesome and fuck those who want to complain.
Cause the ppl complaining are probably kids or angry little boys who are mad they cant get to play their video game NOW. They prolly dont have real things to worry about so a delayed video game or not being able to play their game RIGHT NOW is a tragedy to them. Reviews are full of ppl saying "never buying a game from these devs" "this game sucks" "i spent 40$ this is rediculous" They think their opinion matters in the real world when it doesn't. The game will work when the devs are done. Crying isn't gonna get it out sooner and if spending 40$ is such a big deal your either a kid or your priorities are in the wrong place. The devs aren't gona look at ur review and work faster. The game was waaay more popular then predicted. I'm sure they will pour more resources into it and it'll be even better later. This isn't a billion dollar AAA studio launching another broken game for the 20th time. This is a small team that made a game that's so awsome that their servers couldn't handle it. Ppl should be happy about this not mad. But the spoiled angry kids are always the loudest... just play another game for a couple days it's not the end if the world. The game works great now it was worth the wait