Gameplay Design Rants (tm) - Fail Faster, the Dev Cycle

[nonsonogiucas]

As an introduction, please watch this video about what should be a mantra of game design: "Fail Faster".

The idea behind it is that no idea starts perfect, and less so it's implementation. It is crucial to put an idea, or a game mechanic, to the test as soon as possible, even if that means playing a prototype with very rudimentary graphics and sound fx. Because the faster that idea fails, the sooner you can start, as a designer, to correct it in order to make a better game.

Some games use a dedicated server to test early, even wild experiments. Would Salem development and its community benefit from that? I think it will.

Right in this moment I would much more like to play on an experimental server where game mechanics change every week and balance is tweaked every couple of days if that can help deliver a better Salem MMO sooner, rather than grinding my way up to 100 biles with the current mechanics. But that doesn't mean we should abandon the main server, even if not a lot there are people who like playing Salem as it is right now and it wouldn't be fair nor productive to change their gaming experience too often and too radically.

So the idea is to have two open servers, a main server where long-run progression is being experienced fully and a proper test server where (when appropriate) one would directly jump in the shoes of a 30, 50 or 90 biles character in order to test out wildly different ideas and new content. Like a different world generator for example, would bigger biomes make sense? Would it be fun to have soil purity associated with geographical position? Or maybe different combat moves and weapons, does adding a strafe move or a lasso break combat?

Then, when set of mechanics and tweaks becomes well established and it is pretty clear that the one tested is a better game than the previous one, the new content could be imported (not necessarily wiping) into the main server and an other cycle would start.

Let's try more diverse and new things, let's Fail Faster.

[cannibalkirby]

this might actually be a good idea, mostly all successful games still use a beta test server to keep testing new ideas before implemented, ie League and WoW both use these things, though I would wait until they have things to test

[Feone]

Don't forget that smaller indy game projects like this one also have a shortage of manpower. Maintaining (and creating) a second server takes time and efford. Both of these are very precious to small dev teams.
The Salem community is also spread pretty thin already. Splitting activity between two servers seems like a bad idea.

On the other hand, thorougly testing new concepts is usually a good idea, it would certainly reduce possible bugs and abuses.

[nonsonogiucas]

On the other hand, thorougly testing new concepts is usually a good idea, it would certainly reduce possible bugs and abuses.

It is not much about thorougly testing but being able to test new concepts at all.
From my current point of view (I'm very much open to discussion), having a single server is actually slowing down Salem because the devs will feel the need to over-think every single new idea in fear that even a small mistake could be fatal for the actual community. That in turn raises costs because time is money. The more time you need for a single development cycle the more time it will get to raise the game quality to the point that the player base will start to grow to an acceptable level, the more you'll have to pay for server rent (and your own personal sustenance) before the game really starts to be profitable.

A very simple straightforward example: How are you going to test tweaks in the map generator without a wipe that would risk to totally destroy what little is left of the veteran playerbase?

[Feone]

On the other hand, thorougly testing new concepts is usually a good idea, it would certainly reduce possible bugs and abuses.

It is not much about thorougly testing but being able to test new concepts at all.
From my current point of view (I'm very much open to discussion), having a single server is actually slowing down Salem because the devs will feel the need to over-think every single new idea in fear that even a small mistake could be fatal for the actual community. That in turn raises costs because time is money. The more time you need for a single development cycle the more time it will get to raise the game quality to the point that the player base will start to grow to an acceptable level, the more you'll have to pay for server rent (and your own personal sustenance) before the game really starts to be profitable.

A very simple straightforward example: How are you going to test tweaks in the map generator without a wipe that would risk to totally destroy what little is left of the veteran playerbase?

Tweaking the map gen is probably one of the few things that will most definitely require a wipe and/or test server.

A lot of the inactivity on Salem has been due to the devs though, from what I have seen of jorb and loftar they are not the kind of devs that would care much about upsetting their playerbase with a change in mechanics.
The new team may be different however. They are, appearently, a larger team as well.

Aside from all that, there is a myriad of possible changes that would not neccesarily require a test server or a wipe. Many small additions & tweaks come to mind. From what I have read so far, these kind of changes are the things the new owners wish to start with. If these are succesful and the population of salem goes up then a test server may be a more viable option.

[Orcling]

Who's going to play on that new server? It takes enough time already just maintaining what you have on the server that we have.

[nonsonogiucas]

Who's going to play on that new server? It takes enough time already just maintaining what you have on the server that we have.

Allow me to point out this:

...a proper test server where (when appropriate) one would directly jump in the shoes of a 30, 50 or 90 biles character in order to test out wildly different ideas and new content.

A test server is by definition meant for test. So, technically, no one is going to play on the test server, like no one goes to play on EVE Online's test server. Thus it just makes sense to boost test characters so that they can get to the meat of what they are supposed to test with the lowest effort possible. EVE's test servers strategy for this is Mirroring, you can read about it here, under the heading Mirroring.

Now to reply to your question, players log in a test server for a number of reasons. I could come up with 3:

1) Wanting to help with the development of a game they like by providing feedback on new / tweaked content.
2) Wanting to have their saying on the development process.
3) Wanting to get some early insight on new mechanics or items for strategic purpose.


Interestingly enough, EVE now has 2 test servers, and CCP doesn't even go out of its way to encourage players to participate. Probably because the ones that can prove useful in that regard do not even need to be invited, and the ones who don't...
Also take a look at this, as an example of good use of the community.

[Feone]

A lot of mechanics in this game are very long term. It's not just "Hey let's test this fancy new laser gun, go a few rounds and see if it needs tweaking!"

Issues with most mechanics in this game won't show after weeks, perhaps even months of use by the full community. Too much efford for a test server, even if we had the players to do so. Just skipping ahead won't work because of the more subtle factors. Efford & time required to achieve certain points for example.

[nonsonogiucas]

A lot of mechanics in this game are very long term. It's not just "Hey let's test this fancy new laser gun, go a few rounds and see if it needs tweaking!"

Issues with most mechanics in this game won't show after weeks, perhaps even months of use by the full community. Too much efford for a test server, even if we had the players to do so. Just skipping ahead won't work because of the more subtle factors. Efford & time required to achieve certain points for example.

I'm aware of that...
that is the reason why you actually begin with a beta version of the game.
that is also the reason why you do not want relatively minor things (like balancing combat moves, timing, ui elements...) to change too often in your main beta test server. The main server should remain somewhat stable because its primary purpose is to test long term mechanics.

For example, if you where to rebalance structural damage, would you change that directly on the main server risking that either 1) all structures become indestructible or 2) all structures get wiped out in a matter of days? Look at braziers, Isn't it true most players state that the dramatic change in brazier damage totally reverted the feasibility of base raids?

If you were to introduce a new weapon, maybe just for the item shop, wouldn't you first test it outside the main server to avoid the risk of creating a play-to-win scenario?

Most big companies have internal test servers and thus "could" skip public test servers. Indie just don't have enough people at hand to make extensive internal gameplay tests. The community in our case is testers and player base at the same time. The tricky part in this situation is that if you mess up an update you risk players leaving (are the bells ringing already?) because they invested time in what they perceived as an actual game and not a mere instance of a gameplay test.

[Feone]

A lot of mechanics in this game are very long term. It's not just "Hey let's test this fancy new laser gun, go a few rounds and see if it needs tweaking!"

Issues with most mechanics in this game won't show after weeks, perhaps even months of use by the full community. Too much efford for a test server, even if we had the players to do so. Just skipping ahead won't work because of the more subtle factors. Efford & time required to achieve certain points for example.

I'm aware of that...
that is the reason why you actually begin with a beta version of the game.
that is also the reason why you do not want relatively minor things (like balancing combat moves, timing, ui elements...) to change too often in your main beta test server. The main server should remain somewhat stable because its primary purpose is to test long term mechanics.

For example, if you where to rebalance structural damage, would you change that directly on the main server risking that either 1) all structures become indestructible or 2) all structures get wiped out in a matter of days? Look at braziers, Isn't it true most players state that the dramatic change in brazier damage totally reverted the feasibility of base raids?

If you were to introduce a new weapon, maybe just for the item shop, wouldn't you first test it outside the main server to avoid the risk of creating a play-to-win scenario?

Most big companies have internal test servers and thus "could" skip public test servers. Indie just don't have enough people at hand to make extensive internal gameplay tests. The community in our case is testers and player base at the same time. The tricky part in this situation is that if you mess up an update you risk players leaving (are the bells ringing already?) because they invested time in what they perceived as an actual game and not a mere instance of a gameplay test.

As you said. Salem is in beta, if a change is slightly broken it won't have catastrophic effects. If effects are catastrophic, there's always the option to roll back changes. (At least, I'd hope they make backups frequently.).

The response would be easily tailored to the situation. In your example:
Slightly off balance in either direction? Tweak , effects won't be severe enough to require other actions.
Indestructible? Fix asap, no need for other measures as nobody got their stuff wrecked.
Too easily destroyed? Fix asap, roll back server to pre-patch to undo the large scale damage.