This site uses cookies to improve your browsing experience. By continuing to browse this site you are agreeing to use our cookies. Get more information Dismiss
You're using an unsupported web browser. Some features may not work correctly. Please check out our list of supported browsers for the best experience. Dismiss


Production Dev Blog: Feedback Prioritization

By Julia (nitocris83)
Wed 19 Apr 2017 08:00:00 AM PDT

Producer Vincent Malley brings some insight into the prioritization process of community feedback! For discussions on this topic, please visit the forum thread

Regarding prioritization:

There's no one system that fully covers how we prioritize things. We'll generally give the highest attention to issues that prevent people from playing the game (including getting rewards), or issues that could impact our ability to keep the game running (e.g. Zen Market not working - as distasteful as some folks may feel it is, the game does need to allow players to purchase and spend Zen in order to stay in active development). Then it's a matter of a person in charge of bug triage asking questions like these:


  • As a player, how does this affect me?
  • Would I want to leave the game because of this?
  • Even if I wouldn't leave the game for this specific issue, would it cause long-term dissatisfaction?
  • What portion of the player base is affected, and is there a player-known workaround?



  • Do we understand the issue on a technical level, and is it possible to reproduce this in a controlled environment so we can prove it's fixed?
  • Is it possible to fix in a good way, or do we need to make a quicker, less comprehensive change that may cause more issues down the line?
  • Does the fix risk being worse than the issue itself?
  • Does the fix fit within a new build's test cycle?  Can we even get it in the current module's updates?
  • Do we have a better, more comprehensive fix coming in a future module?  If so, is it soon enough that we can let this issue slide for the time being?


Basically, if there's a question that'd affect how you'd prioritize an issue if you were in our shoes, it's probably one that we ask ourselves, or one that can contribute to prioritization of a bug, feature, or piece of feedback.

There's a human element in triaging these issues, too. These questions may have different weight to different people - we all have our different ways of playing and perceiving the game, and some of us are more knowledgeable than others about certain content, certain reward systems, certain UI flows, etc. If we have questions we can't answer, we'll go talk to a team member who knows - but sometimes those team members are out sick, on vacation, or only know part of an issue.

Even a difference in wording can affect how some of those questions are answered. As an example of that, see "Great Weapon Fighter: Unstoppable can become temporarily unusable during combat" vs. "Great Weapon Fighter: When activating Unstoppable right before gaining Determination, player gets locked out of using it again." An open-ended mystery can sink days of development time, but the right tidbit of information can change an issue from "I have no clue where to even start" to "Oh, I could try this change, and if that fails then I've at least ruled out a good hypothesis."

There's also the fun aspect of: No matter how well we may know the game, the players know it better. Our knowledge of the game is innately affected by us seeing how it works on the back end, and our perception is biased toward the knowledge of how it "should" work, rather than potential emergent factors.


Regarding how we get / escalate reports and feedback:

Our info can come from many sources. Often before a module's release, it comes from our Quality Assurance team (colloquially known as QA or testers), team-wide playtests, or feedback threads on the Preview forum. After a module's release, it more frequently comes from trends identified by Customer Support, from our Community team (often escalated by Nitocris83 / Julia and several others), and from the forums themselves, where they can be spotted and escalated by myself or other developers who browse (if Julia hadn't gotten to them first).

If I see a bug report posted on the forums, myself, I often write up a quick summary and a potential first shot at repro steps to our QA team, and after they verify it, it gets written up officially in our bug database. This is where the wording of player posts comes into play - I have a "jack of all trades" working knowledge of the game and have a loose understanding of how most of the systems work behind the scenes, so if a player reports to me, as an example, "When using any Artifact Off-Hand, if I rank up the Aspect of the Pack power to Rank 4, the Class Feature effect stops working on the power" it's relatively easy for me to say, "Oh, this is probably an issue with some prerequisite in a segment of the power's setup," and it's then an achievable task to translate that into terms our developers will be able to act off of without significant extra research. If a bug report is instead, to use a hyperbolic example I haven't actually seen, "AotP is broken," then it's unclear where to start looking, assuming I even know what that acronym is - and if our QA team is already swamped, I'm not going to be able to escalate this to them in good conscience.

Still, I'm often willing to do some research, depending on the perceived severity of the issue (or, since I'm human, the allure of a mystery if I have enough of a hunch of how to figure it out). It just heavily depends on how much time I have available between other things I need to get done.

Feedback is much the same, even though cause and effect are a little squishier. "I don't like X" and "I like Y" are both important for us to hear, but it becomes much easier to focus on what's working and what isn't when we understand the individual parts involved. "I don't like the Arcanic Focus grind right now, because it feels like the game's telling me I need to grind dig sites for 12 hours a day to make standard campaign progress" is a great piece of feedback, because it tells us the perspective from which folks are speaking, the goal to which they want the Arcanic Focus (campaign progression), and alludes to potential bugs or design flaws in drop rates.

There will always be more reports than we can field, but we could always do better and improve on our communication and responses for these. I know that, personally, I have some weeks where I'm on the ball for responses to bug reports, and some weeks where I can barely find the time to review the forum at all.


TL;DR for priority: Prioritization is difficult and not a science.  If it keeps people from playing the game or supporting us, that will likely see the most attention.  The rest is a judgment call based on questions we ask ourselves, and risk / time taken vs. reward.

TL;DR for escalation: When reporting bugs and feedback, it makes it much easier for me to escalate when there's a clear cause and effect - but don't let that stop you from reporting issues for which you don't have that information, if you think the severity is bad enough. Also remember that we devs aren't always up to date on the standard abbreviations.


Vincent Malley

nw-news, nw-launcher,

Most Recent More

Earn double Stronghold Shards and get 15% off VIP!
Read more
It's a glorious weekend for PvP!
Read more
What wonders does the Mysterious Crystal Charger hold?
Read more

hover media query supported