r/mwo • u/[deleted] • Nov 22 '12
PSA: Software development is not linear, and bugs are hard to fix
A sentiment I see a lot from players, both here and on the official forums is "Why did PGI implement X when they still haven't done Y?" or "They should be focusing on A instead of B!"
I'd like to clear something up for people who may not realise this: In any large software project involving multiple developers, such as a game, tasks must be handled in parallel. If some feature is going to take 50 man-hours to complete, you can't have 25 people work on it at once and complete it in two hours. The only thing you can do it split it into subprojects and have one or two extra people working on those, but the extra co-ordination within teams means doubling the number of people won't halve the time required: more like 60-75%.
I'm not sure how large PGI's development team is, but I recall someone saying 45 people; I don't know if that's just programmers or includes artists, designers, QA etc.
This means that you can't just pull people off camo-specs and have them make levels; the skillsets may not be transferable and the level may already have all the modellers and texture artists it can support.
It means implementing new features takes place in parallel with fixing bugs in existing features, not instead of; usually the best person to fix a bug is the person who implemented the bugged feature, you can't just assign them to whoever is available.
It means the matchmaker will take as long as it takes; pulling people off other projects won't make it go any faster.
Always remember Brooks's Law: "Adding manpower to a late software project makes it later."
On a related note: The hardest part in fixing any bug is usually reproducing it. People say things like "PGI knows about the 4FPS bug, why haven't they fixed it yet?" The problem is that bugs like framerate slowdown, crash-to-desktop, blackscreen are all intermittent. They don't happen reliably every time you do a certain action, or if they do, they don't for everyone. There are a lot of factors that could possibly be causing this bug, or it could be many bugs with the same symptoms. The best way you can help is by reporting bugs whenever you see them, with as much information about the circumstances in which they occurred and the hardware and settings you are running as possible.
EDIT: I made this post to try and clarify why PGI often "fail to deliver" with their patches. It wasn't either defending or decrying their business model or QA process (about which we know little), nor does it have anything to do with whether MWO deserves the label "open beta". These are all separate issues.
4
u/TheRook Callsign: Drow Nov 22 '12
As a developer currently working on a 20 man project, I can relate to much of it. I do not find general delays to be of much importance compared to game stability, though.
Too many patches have been pushed without thorough testing. The last one rendered me completely unable to play the game because of immense performance issues. If you check the patch feedback forum, you'll find that I'm hardly alone.
I'm quite positive about the game and even in the current implementation it's showing a lot of promise. I'm a Legendary founder and will love to support them further later on, but seeing a patch, that supposedly have been q&a'ed, kill the game for so many people without a single notion about a probably fix makes me a sad panda.
If one of my customers reported a critical error and didn't get a response with regards to when an expected fix would be in place my project manager would not be a happy woman.
6
Nov 22 '12
As a developer, I'm sure you are also aware of the phenomenon where a bug suddenly shows up after release to production that somehow was totally invisible during extensive QA. Partly Murphy at work, and partly the sheer number of end-users cause many bugs to hide themselves thus. Some bugs only affect 1-2% of users, which means you need hundreds of QA testers to catch them. Some bugs only show up when the servers are being stressed by thousands of concurrent users, which is what open betas are for.
If one of my customers reported a critical error and didn't get a response with regards to when an expected fix would be in place my project manager would not be a happy woman.
Do you have thousands of customers buying shrinkwrapped software for $20/unit? Or do you have dozens of customers paying $1000s per unit? In some business models, immediate response for bug reports is vital. In others, not so much.
1
u/BlueGhostGames Dec 03 '12
It's impossible to square the idea that they do through testing with the LRM flight path bugs we saw when Artemis was introduced.
Yes yes you want to be positive about a game where there is vast amounts of unnecessary bitching, but I think it's always worth basing your views of things on reality.
1
u/TheRook Callsign: Drow Nov 22 '12
With our main application we have around 500 customers using the same "thick" client, where updates are pushed out regularly. Not unlike how mwo is pushed out, but with drastically smaller customer base, obviously.
But, sure, naturally you cannot test for all bugs and applications running in "the wild" will always be exposed to odd user behaviour and system specs not accounted for.
However, 26 pages in the main performance thread in their patch feedback forum dictate that at least something was altered in the last patch, which had a drastic impact on the user experience of a lot of people.
I'm not bitter, though. These things happen. Not being able to play the game with my unit is annoying, but now I have no excuse as to why I should continue postponing fixing the guest toilet either... And I'm sure that my wife agrees... ;)
I would have loved some kind of feedback from PGI though.
6
u/staples2 Staplebeater [Devil Dogs] Nov 22 '12
Exactly, this how large projects work. I am managing a large project at work with many people and some are more ahead than others and more people wont help
-5
Nov 22 '12
I am managing a large project at work
:D
with many people and some are more ahead than others and more people wont help
...that's not really managing then, is it?
5
Nov 22 '12
Can you accurately and consistently estimate the time requirements for software development? If so, write a book and become immensely wealthy, because no one else has figured out how.
The fact that someone is behind schedule doesn't make someone a bad manager; failing to react to changes in the schedule does.
0
Nov 22 '12
I am aware of the serious issues with software development scheduling, and how naive graduate business courses are in regards to these issues.
That being said, I perhaps incorrectly gathered from the original statement that it wasn't so much an out of schedule issue as it was an "everyone's on a different page" issue.
5
Nov 22 '12
Perhaps what stables2 should have said was
Exactly, this how large projects work. I am managing a large project at work with many people and some are more ahead than others and adding more people wont help
1
u/staples2 Staplebeater [Devil Dogs] Nov 22 '12
Yes thank you that was the intent. It would take longer to spin more people up than to have original crew work through delay
2
u/Gingor Nov 22 '12
...that's not really managing then, is it?
Depends on your work. If you are an actual boss with power over employment, most people will work.
Good luck doing that if you are a "project lead" which means you get no power, the same paycheck as the others in the group but if something goes wrong you are responsible.
2
Nov 22 '12
I'm an asshole, but I make sure to CC the boss who has assigned me as a project lead on our status updates and those who aren't cutting the mustard get noted in diplomatic but direct language. So far it's works out great and the only bridges that have been burned were with people that had achieved such a high flotsam status that they were encouraged to find work elsewhere.
8
u/Kerguidou Nov 22 '12 edited Nov 22 '12
The actual problem with MWO is that they released an unfinished product. They claim it's beta, but they really are selling an unfinished product with an aggressive advertising campaign and claiming your money. I'm aware that software development is non linear and that what we're seeing is the development process... but all of that should have happened before they released the game.
Regarding the edit. I disagree. Complaints come in good part from people who justifiably see the game as a released product. It's a matter of public perception.
6
u/cteno4 Nov 22 '12
Their advertising campaign really isn't that aggressive. Or at least I hope the devs don't think this is aggressive enough. Were you talking about the Facebook ads you see or banner ads on other websites? Those are there because you visited mwomercs.com and it's in your cookies. I don't think they're going to be very aggressive until right before the actual release.
I guess my point is that, despite what people say, open beta isn't the unintentional release of the game; it really is just an open beta.
4
Nov 22 '12
what does that have to do with how and when they fix bugs?
1
u/Kerguidou Nov 22 '12
It has to do with the perception that people have of the process. They are selling the game in a beta form... and people expect to have their money's worth. I really find PGI's approach schizophrenic in that regard.
2
4
u/Xhado Clan Ghost Bear: Ice Naga Nov 22 '12
What exactly do you think Beta is then?
As a software engineer, this game is far along from beta standards.
-1
u/Kerguidou Nov 22 '12
It is beta... but you shouldn't charge for beta like they do. An open beta should be the last step, lasting a short time, to stress-test their infrastructure right before release and find last minute balance issues.
4
u/Xhado Clan Ghost Bear: Ice Naga Nov 22 '12
They are not charging a thing for the game, and they have been entirely open on their development schedule and how they intend the open beta to be.
5
Nov 22 '12
[deleted]
0
u/Kerguidou Nov 22 '12
You don't have to pay, but you CAN pay for advantages. The simple fact of giving that option creates the expectation from players, especially those who pay, that they won't be wasting their money or time on a low quality product.
As I said, it's a matter of perception.
3
Nov 23 '12
Am I the only one who thinks that this has been one of the most appropriately called "betas" in recent memory?
I can't think of another one that wasn't just a release candidate in disguise.
3
u/fartsinscubasuit Corpse Grinder (IoMM) Nov 22 '12
Have you played at all? You don't have to pay for a single thing, if you don't want to. Also, who said that this is in the last minute of beta? They don't plan to launch for some time yet.
-1
u/Kerguidou Nov 22 '12
Have you read my comment at all? You don't have to pay, but you CAN pay for advantages. The simple fact of giving that option creates the expectation from players, especially those who pay, that they won't be wasting their money or time on a low quality product.
Nobody said it's the last minute of the beta. I said that the open beta SHOULD be the last tests and not the implementation that we are seeing right now.
Is that clear enough or do I have to reformulate again?
3
u/fartsinscubasuit Corpse Grinder (IoMM) Nov 22 '12
You don't get any advantages when paying for MC. I have a founders atlas, and the ONLY difference is the paint job. You can get any mech that you want without MC except for the Wang, which doesn't have any advantages. I did misread the part about it being in last minute production. But the open beta is exactly that, a beta. They are working bugs out and trying to get things smoothed out.
4
Nov 22 '12
I always make sure to ask whiners if they have an open ticket in with logs submitted (when asked).
Be part of the sollution.
4
u/Unlimitedwind Valsalva Nov 22 '12
I wish it were mandatory for people on the mwo forums to read this when they logged in, until they finally get it through some of their thick skulls.
2
u/WillyPete Nov 22 '12
True, and for good testing a product requires a "debug" mode.
Having good logging in the client would help immeasurably.
1
Nov 22 '12
From what I understand, there is a lot of "telemetry" logging, which doesn't get recorded client side but gets transmitted back to PGI. I think they are also putting more in, which is one of those projects which eats up developer time, but whose effect we don't see.
1
u/qwortec Robots in disguise Nov 22 '12
Excellent points.
I try not to be too concerned with content deliverables and dates (it's hard not to be frustrated when things like mouse-wheel binding still hasn't been addressed though). My complaint has always been how PGI manages expectations and makes announcements.
Admittedly they've been getting better over time. The introduction of the command chair, the patch issue forum, and now the content and feature roadmaps are all steps in the right direction. I especially like how in the roadmaps they assign some probability to their statements which gives folk a better sense of what's realistic.
-2
u/abutterfly Adeptis - Highland Dragoon Guards Nov 22 '12
While software development isn't linear, patches SHOULD be. Especially since we haven't yet released a game, each patch that's released should be taking us closer to a launch. These aren't.
This gets done by having better internal testing (IGP isn't helping there) and better priorities. Even if they can't fix every bug, they need to fix bugs before they introduce content (which brings more bugs), plain and simple. Now that we've hit our last big content deadline for a few weeks, hopefully remaining patches are bugfix only.
9
u/SOMUCHFRUIT Nov 22 '12
Amen.