[PMC] Brainstorming on Improving the Mod Selection Process

[Mumberthrax]

In the fall Mod Brainstorming thread a short while ago I posted a comment suggesting we approach the process for brainstorming moderators differently than they have been in recent times. In our staff meeting it was suggested we move the discussion on how the process could be improved to its own thread, so here it is.
 
What follows are my thoughts on the process we have now, and then my suggestions. If you just want the canned version, open this spoiler:
 

The mod brainstorming threads have traditionally been used for staff to offer an opinion on different players who have been suggested additions to staff. Lately they've had more of a feeling of being used as voting threads, with brief blurbs about why or why not. I have a problem with this casual setup because it seems like it benefits those who are popular or friends with vocal moderators - even if those players might not actually be appropriate for staff, and it also tends to lend itself to ignoring quiet players or those who are not good friends with any current or vocal staff. It also lends itself to an echo chamber effect, where one person offers a positive or negative opinion on a proposed addition to staff, and others in the discussion who may be neutral or have little knowledge of the player in question offer an opinion and an effective vote based solely on the input of one other discussion participant.

My proposal is to have a set of focused questions that we ask ourselves when evaluating players for addition to staff, rather than a vague yes/no. We don't have to do that for all of them neccesarily, but i believe each question should be taken into account when offering an opinion on the player

 

Challenges I've identified:

• Voting, not brainstorming
  
  The mod brainstorming threads have ever more been resembling simple voting threads. Comment replies are usually a list of names that have been mentioned so far, a yes or no, and optionally a brief sentence saying a reason why. This is a problem because it shows us not quantifiable information about the player in question, but rather mostly only information about the moderator making the comment.

• Social dynamics of a forum thread discussion
  
  (e.g. influencing others' "vote" with one-off comments, unsubstantiated claims, or vague opinions). As these are the reddit public minecraft servers, I'm sure most of you are at least somewhat familiar with reddit's voting system, and how one downvote on a comment can seem to cause many more to pile on, while one upvote can seem to cause a comment or post to rise to the top even if it is only mildly good. Reddit's actual voting mechanisms aside, the basic idea I want to express with this image is people allowing their opinions to be influenced solely by the fact that someone else has an opinion on the matter.
  
  If I have a friend who I want to be on staff, I may say very good things about them and ignore or brush aside any negative qualities. If others are neutral or do not speak up quickly, those without opinions may come to adopt the opinions I've expressed.
  If there is someone I hold a grudge against, I may make critical remarks, or even make something up about them and claim it is true without evidence, and others who would otherwise be neutral on the player may decide to vote "no".
  While the very basics of such a strategy could be functional, there is obvious room for improvement.

• Staff who we later learned should not be staff
  
  This is a sensitive subject, and it is still something that needs to have attention on it. We've removed a few staff members in the past for either betraying our trust, or demonstrating a lack of maturity, a lack of responsibility with the tools given them as staff, or demonstrating outright malice for players or staff members.

• Favoring friends and missing potential gems
  
  There are doubtless players on our servers who are moderator material whose names have not been submitted to us, who maybe chat primarily in their clanchat or play on odd hours of the day so we don't notice them as easily. Our process thus far favors names of players who are friends with current staff, and unfortunately does not favor those who are not close to current staff or are not popular. nerd.nu/applyformod was one tool created to attempt to resolve this, and it is underutilized and under-advertised.

Are there other challenges for us to overcome here? Other issues that need resolving with the process of adding new moderators?
 
Solutions:

Having thought a little bit about this, I presented some of my thoughts on possible solutions in the fall mod brainstorming thread. The summary of which is at the top of this post in the first spoiler. I'll stick a sort of full rundown of my ideas here in a spoiler too, so it doesn't get unwieldy.
 

Begin by simply aggregating names. Use nerd.nu/applyformod, names of people we individually think could be good, those who have sought us out to share names or volunteer. Much the same way as we always have done. But not voting, just collecting the names into a list.
 
Then, cull the names that are easily removed. Those players who are inactive on the servers or our other services and those who have demonstrated a willful disregard for the rules or disrespect toward staff or other players (I could name a handful of such players off of the applyformod list right now). Remove past staff from the list, as re-modding is an entirely separate process handled by head admins (discussions on that process should be separate from this one).
 
Next we have a few options.
 
I propose we ask ourselves a series of quantifiable questions about the players in our list. Something that could be kept in a sort of spreadsheet or table. things like:

• How active is the player on the different branches of the community (i.e. servers, forums, subreddit, IRC, Mumble, etc.)?
• What server(s) does the player currently frequent?
• How long has the player been a part of nerd.nu?

From these questions, we can consider whether the player seems like they would be an active moderator, where the player would most likely be moderating, and whether they have been around long enough to be familiar with our rules and the basic responsibilities of moderators. If they are very new, we might go ahead and set the name aside until later. If they are only active on mumble or the subreddit for example, we should keep that in mind as we continue.
 
After this, we can look at any red flags or sources of possible confusion by asking ourselves questions such as:

• Does the player have any alternate accounts or aliases?
• Has the player harmed the community to a publicly visible degree? If so, how and to what effect?
• Has the player been banned before? If so, what was/were the reason(s)?
• If the player has had notes added, what were the reasons?
• Has the player demonstrated an observable disrespect for staff, the servers, or other players?

Or similar questions that seek to identify any warning signs. If any of these turn out to show the player has a bad attitude or a disregard for the rules, or is an alt account of a known troublemaker, then we may not need to continue examining them.
 
And lastly questions that seek to identify the observable positive qualities of the player:

• Has the player made any notable contributions to the community? If so, what were the contributions and what effect did they have?
• Does the player appear to care about the community?
• Has the player demonstrated a reasonable level of maturity and/or responsibility?

If we do not know the answers to these questions, then I suggest we take time to observe the player in question, perhaps get to know them.
 
Not all of these must be answered outright for every single name on our list of potential candidates. I think it would be simplest if we had a spreadsheet or table in one place to keep the information. However if we wish to simply do an adaptation of our current process rather than a whole new system, we could at least attempt to keep these sorts of questions in mind when writing our little blurb about each name, and hopefully that will lead to a more objective and thorough evaluation for each proposed candidate, or at least elevate the perception that a simple "yes" or "no" is meaningful or useful.

 

example:

John Doe is a relatively new player to our servers, spends most of his time on survival, but has been active in a few discussion threads on the servers in general too. Not sure about any alt accounts, I guess that'd be an admin thing, but he does have a note about crop grief made by BobSmith about two months ago. I don't see any past bans in the appeals forum either. Other than the note, I haven't heard any critical comments about him, no problems with staff or other players. As for good things, he made an awesome monument near spawn this rev, made it open access and put a lot of attention to detail in it. He's been helpful explaining to new players about how modreqs and safebuckets work. I am a little concerned about some off-color jokes he's made about adult topics, but its never gotten out of hand or anything. Overall I think he could make a good addition to staff. Someone to keep an eye on at least.

 

Ok obviously I was advocating in detail for our John Doe, and I was pretty familiar with him and his behavior on the server. My point is that the questions don't have to be answered in a tabled form necessarily - they can be taken into consideration to give a more thorough evaluation in our little blurbs about players. Other players who may not know John Doe as well might follow in a shorter fashion:

 

Regarding John Doe, sounds like mumber got the info on his activity right. I don't play on S much so I don't see John really at all in-game, but I've seen his comments on the forums. I will say though that I have heard players in mumble complain about his tendency to spam clanchat. Maybe something to look into? I don't have anything to add on the other focus questions, looks like we've already got the basic ones, and again I don't know him well enough to say about positive or negative qualities. I can try to talk to the players who said something about clanchat though and find out more

 

I dunno. I mean, in this response the moderator doesn't offer much in the way of additional answers to the questions I posted above, but there really isn't a lot they *can* say since they don't know him. The main point is that this is made aware to everyone, that our second respondent has no familiarity with the moderator, but there is at least some novel information here about clanchat spam concerns and it's more than a short "I don't know him". Mods wouldn't be obligated to say a ton about each name if they literally have nothing to add, but as long as *someone* answers some of the questions, and it's open for anyone to offer alternative answers to those specific questions, I think that's better than what we've got.
 
After this process, the head admins do their thing, look at the brainstorming thread's discussion, put together a list of names for mods to give a final vote on - while mods should know that this vote does not necessarily carry more weight than the brainstorming thread itself when it comes to head admins' decision to invite a person to become a mod or not, it does allow moderators to offer a final opinion or position or to change their mind before a decision is made.
 
Then heads take allll this discussion and the last vote, decide who they believe would make good additions to staff, speak to them and invite them. Once each has accepted or declined, an announcement post is made, and we move onto training which is an issue that needs its own post.

 
I've identified some possible objections with some of these ideas:
 
1) It might be too much work
2) It is a bit creepy to have an HR sheet on players
3) Our current system works well enough

 

Now that I've shared my walls of text, what do you guys think? What other issues are there for us to overcome in the process of adding new moderators? What alternative solutions are there? What do you think of the ideas I've presented - and do you have any thoughts on how to streamline them or improve them? Is all of this a waste of time and energy?

 

Credit where credit is due, eehee2000 is the one who messaged me with this idea a while back.

Edited November 16, 2014 by Mumberthrax
typo

[Trooprm32]

I like the idea of having a list and running through them to eliminate names that are clearly not suitable for nomination, and then going into more detail with the names that make it through.

[buzzie71]

I think a variation of our current system is the best way to go - what is most needed now, and most lacking in the current system, is really just the discussion aspect: explaining in depth about particular players, making a case for or against.  The questions you put forth are definitely good starting points for the additional explanation with each names,  even if it's really just pointing to other posts and agreeing with them.

[TornadoHorse]

I don't have too much of a problem with how the mod nomination process is right now. With the 4 challenges that you stated at the top of your post, I think all have very simple solutions. 

 

• Voting, not brainstorming
• Social dynamics of a forum thread discussion
• Staff who we later learned should not be staff
• Favoring friends and missing potential gems

 

1. If you comment, give a reason why you do or don't support a player. When I comment if I support someone then i'll talk about the good things they've done, what they'd add to staff. If not then I usually add a reason. On those who I don't know I don't want my vote to impact other's opinions so I say "neutral". I think we pretty much do this already with only a few not giving reasons. If you don't give a reason, don't let that vote influence the results.

 

2. I don't fully understand this. If I had seen someone doing something shady that made me believe they wouldn't be great on staff then I think I'm totally allowed to bring that up. I did this a few rounds ago with a player however I didn't present any evidence to support my claim. It was later found out that the player was kidding about this. In future I will try not to sound so influential about that I kind of thing? However I don't want to have to be presenting evidence for every opinion I have about a player, that would be way too much work for everyone.

 

3. Make it clear to players that when they are given the responsibility of staff that if they aren't fulfilling that responsibility then they will be spoken to and in the worst case scenario, demodded.

 

4. I don't think this is an issue, not in the rounds that I've seen. There shouldn't be any competition between players to get moderator, if 2 players are both very good candidates for the role then both should be added. I'm not sure so sure about P and C, but I don't think this has been an issue on S for a long time. I nominate those who I think would be good regardless whether I'm around them all of the time or if I rarely speak to them, I know many others do the same.

 

I think the changes proposed would create too much work for not that much of a gain, if that makes sense? I think if we were to change anything then it should be to do with the applyformod page. Many large servers use staff applications to see who would like to become staff. This might not work as well here but I don't think it would harm the servers in any way. The players would be able to provide information about themselves and how they would think they'd help the servers. Downsides of this would be that becoming staff may seem like even more of a promotion than it is now, I wouldn't like to promote that idea. If someone has applied for mod then speaking to them about it may help. If they aren't chosen, let them know why so they have something to work on and aren't left feeling totally ignored, this has happened frequently in the recent past. Speak to them about why they think they would be a good addition to the staff team. I think communication is definitely something we could work on in many of our problems.

[TheAcademician]

I had some thoughts on this and Mumber asked me to share them here, so I will attempt to do so.

 

The current moderator nomination system isn't broken, but it certainly seems to be less effective than it really could be due to the nature of how it is done. Every few months we combine and toss names around en masse and try to come up with a list of potential candidates. This usually leads to a very bloated thread where it becomes very easy to lose track of who is being discussed and what names are in contention.

 

My proposal would be to do away with the mass nomination thread and instead utilize a sub-forum to PMC that would allow anyone, at any time, to nominate a player for a moderator position. Discussion would then center around 1 person per thread, and people could provide testimony and evidence of their reasoning for/against the candidate. Server admins could also post criteria they are looking for specifically at the present time like, "We really need more moderators during X time, please focus on finding someone who is active during this time that may be valuable."

 

Anyway, those are my thoughts :D

[WayneByNumbers]

I had some thoughts on this and Mumber asked me to share them here, so I will attempt to do so.

 

The current moderator nomination system isn't broken, but it certainly seems to be less effective than it really could be due to the nature of how it is done. Every few months we combine and toss names around en masse and try to come up with a list of potential candidates. This usually leads to a very bloated thread where it becomes very easy to lose track of who is being discussed and what names are in contention.

 

My proposal would be to do away with the mass nomination thread and instead utilize a sub-forum to PMC that would allow anyone, at any time, to nominate a player for a moderator position. Discussion would then center around 1 person per thread, and people could provide testimony and evidence of their reasoning for/against the candidate. Server admins could also post criteria they are looking for specifically at the present time like, "We really need more moderators during X time, please focus on finding someone who is active during this time that may be valuable."

 

Anyway, those are my thoughts :D

 

I was just about to suggest myself that, after the initial "name-triage" removing the obvious cases, each name be discussed one by one rather than an open floor.  This, however, could result in some quite long threads indeed if we have to finish one discussion to start another.  I like your idea much better, allowing more specific and informative/informed discussion for each candidate simultaneously.

 

As a small detail, if we're going to be making a list of such "character evaluations," a sticky post on top reminding people that future mods will have access to their own evaluations (and other decorum guidelines) might keep things a little more on the civil side.

[buzzie71]

I had some thoughts on this and Mumber asked me to share them here, so I will attempt to do so.

 

The current moderator nomination system isn't broken, but it certainly seems to be less effective than it really could be due to the nature of how it is done. Every few months we combine and toss names around en masse and try to come up with a list of potential candidates. This usually leads to a very bloated thread where it becomes very easy to lose track of who is being discussed and what names are in contention.

 

My proposal would be to do away with the mass nomination thread and instead utilize a sub-forum to PMC that would allow anyone, at any time, to nominate a player for a moderator position. Discussion would then center around 1 person per thread, and people could provide testimony and evidence of their reasoning for/against the candidate. Server admins could also post criteria they are looking for specifically at the present time like, "We really need more moderators during X time, please focus on finding someone who is active during this time that may be valuable."

 

Anyway, those are my thoughts :D

 

Certainly the advantages of this are clear if a mod nomination thread runs for a long period of time (as has happened before - a quick check through PMC showed a thread that extended for four months and seven pages apparently O.o).  Worth a shot :D