Policy Review Policy Review - Polling Process

[Deck Knight]

Personally I think this is a horrible idea. Gauging support is one of the BIGGEST parts of moving submissions from discussion to the actual poll. There were parts of CAP 5 where I had to go on judgement alone along with support from users. Another huge part of things like movepool (and really any other submission poll) is that ALL submitters have the ability to respond to feedback, whether it's on IRC, the Server, or on the forums. That wouldn't be allowed here because you're limiting the democratic process in a sense. In congress, representatives don't just blindly submit bills to the speaker, and vote on them. They have a chance to defend their proposition and make changes accordingly. With this, there is no room for that crucial debate portion, except by proxy.

Congress is a horrible analogy; half the crap gets submitted in complete darkness in the majority party's private committee meeting, and then gets shoved through before anyone at all gets a chance to look at it.

Another reason why I don't support this proposal is because it puts a ton of strain on the TL. TLs already take enough crap from everyone when they don't put so and so's submission in. Some TLs could feasibly rig the poll, and nobody would know the difference if they can't reveal the submitter!

People know their own spreads. If their final product has been tampered with then they will know about it, and can PM a mod about it. As far as TL's "rigging" a poll, have we ever nominated a TL malevolent enough to even try and pull that? I sincerely doubt it. The vast majority of experienced CAP contributors are either too mellow to care or too noble to tamper.

If it's too much strain on the TL, as hydrolphin recommended the ATL could be responsible for receiving the submissions and updating them, which would give the ATL a duty other than waiting for the TL to fall ill/get grounded/spontaneously combust.

Finally, a lot of you are assuming that the people you recognize most don't make good submissions, when in reality that's usually the reason people are recognized here, for good ideas. This is besides the fact that a "best submission" is completely subjective, and as we all know, everyone in CAP has their own ideas about how a project should turn out.

It's not so much that experienced users don't make quality submissions, its that generally speaking experienced users (server mods, serial poll contributors [me]) tend to get the bulk of the support between them.

Finally, nothing specifically precludes anyone from mentioning their spread to their friends in private messaging. You simply have to trust them enough not to spill the beans in the topic. Experienced CAP contributors will know enough not to be shouting any privy info like that from the hilltops.

 

[Onii]

If someone votes without looking at the submissions now, why would they be compelled to look at them without names? In this case, putting numbers to the submission is no better than the case we have now.

I think darkie has a point here. This would be a good idea if reputation had little or no correlation with the quality of the output. But the primary reason that users gain a reputation in the first place is through a history of quality contributions. Hence, it is more likely that this group will produce a quality contribution compared to the average user.

Also, voting by reputation generally does not and should not happen with sensible users. And as for the "army of fanboys" that may possibly skew the vote with such voting, I would much prefer they vote on reputation rather than on random numbers, because, as stated above, it is more likely that they will pick a quality submission.

 

[jagged_angel]

Also, voting by reputation generally does not and should not happen with sensible users. And as for the "army of fanboys" that may possibly skew the vote with such voting, I would much prefer they vote on reputation rather than on random numbers, because, as stated above, it is more likely that they will pick a quality submission.

I quite agree that sensible users would hopefully ignore usernames in order to vote without bias. Unfortunately, in any community, friendships and fandoms develop, as do feuds, and these most certainly do affect voting outcomes.

As for the reputation vs random numbers - both of those would be equally bad but if the names are removed, even the most noobish voter won't just Randomly pick a number and vote for it - who could see a point to that? They will either read the submissions and form an opinion based on the discussion and their own views, or they will not vote at all.

Either of these outcomes is better than going "Ooh hey it's <Insert Favourite User Here> I'll vote for him! Besides, I've never even heard of the other guys..."

 

[Kaxtar]

Why can't I look at the submission? Your descriptions are completely unobjective. If someone votes without looking at the submissions now, why would they be compelled to look at them without names? In this case, putting numbers to the submission is no better than the case we have now.



What if that option was winning with a VAST majority of the vote? Is it fair to the community to strike down an option that literally everybody wants just because someone tells which option is theirs?

You can't look at the submission because it is a hypothetical case. You have a point about them not looking at the submission. And your right, numbers is not a whole lot better than we have now

And if the option was winning by a vast majority, that person should have no reason to blurt out their name. If they do, however, and we should be prepared for all knids of scenarios. I have an idea, however harsh it might be. The poll could automatically close, and a new poll could be opened with all options but the exposed one. This solution is not ideal, but I can't really come up with a better one without it being unfair to those who followed the rules of polling blind. And the only people who could expose the real name are the TL and poster of the submission. The submission poster already has a plan above of what should be done to the poll. God forbid our TL wrecking a poll.

As I said, my "fixes" are not by any means perfect, and if someone could come up with better options, it would be greatly appreciated.

 

[darkie]

You can't look at the submission because it is a hypothetical case. You have a point about them not looking at the submission.

That's not what I'm saying. In your hypothetical case, the two situations are vastly different and therefore incomparable. In your situation one, the voter votes simply on name, without looking at the submissions. Yet in your situation two, the voter takes the time to read the submissions. Why is he compelled to look at them between the two situations? In the first situation, the voter will vote for the choice which will probably not be innately bad. "it is a hypothetical case" is not a valid answer for answering why, lol, because anyone can make up a "hypothetical case" that makes their point seem relevant.

And if the option was winning by a vast majority, that person should have no reason to blurt out their name. If they do, however, and we should be prepared for all knids of scenarios. I have an idea, however harsh it might be. The poll could automatically close, and a new poll could be opened with all options but the exposed one. This solution is not ideal, but I can't really come up with a better one without it being unfair to those who followed the rules of polling blind. And the only people who could expose the real name are the TL and poster of the submission. The submission poster already has a plan above of what should be done to the poll. God forbid our TL wrecking a poll.

Again, that's not what I meant by "fair to the community". Say, if tennisace's option is winning with 100% of the vote. He says, "thanks everyone for voting for me!" Do we punish the community by excluding an option that literally everyone wants? How is that fair at all to the people who voted, wanting tennisace's submission.

 

[Kaxtar]

I guess you misunderstood what I was trying to get across. I am saying that in Situation 1 that you Can't look at the submissions, not that our voter is too lazy. Knowing the more famous members of the community, and without being able to look at the actual submission, this intelligent voter votes logically for X-Act, as he has made and is still making many intelligent, useful resources and such for the community. Kaxtar, on the other hand, he/she has never herd of before. So he/she comes to the conclusion of "I'd rather vote for a genuis than a noob", and votes for the genuis because he can't see what he is voting for.

These two situations are different, because it can help to understand the effects of one situation if you can start by looking by their oppisite. So they are not exactly comparable, but by looking at one you can help to see the other more clearly, and that's exactly why Situation 1 is there. Now Situation 2 is vastly different, because one is the reverse of the other. Now the situation is that "if I can only see what I'm voting for, but not who posted it, I should vote for the one that appeals to me more." So in this hypothetical case, he ends up voting for Kaxtar over X-Act because, for some reason, he likes mine over X-Act's.

About tennisace's submission, he is not an idoit nor a ass, and so will watch his mouth and make sure he does not accidentaly blurt out "thanks everyone for voting for me!", or something amoung those lines. And any community member who is intelligent enough to make a submission that everyone falls in love with will not allow a slip-up of that kind to happen. And I suppose, if the submission has been blowing away everyone elses', exceptions could be made. And if you come up with a better option, by all means let us discuss it.

And also, I would like that if we were to continue this argument, Darkie, that we do so via PM. We have already clogged up this PR thread enough as it is.

 

[billymills]

What if that option was winning with a VAST majority of the vote? Is it fair to the community to strike down an option that literally everybody wants just because someone tells which option is theirs?

I don't care about people claiming which submission is theirs. Maybe both parties could claim it was theirs etc. There is no problem with them claiming it's theirs, as long as it's not in the poll at the top. If the names are in the poll at the top, people choose them based on names. If people scroll down to see who submitted sub1 and sub2 (on post 4 and 5 for instance), they likely read something about them. At least I'd hope they would.

Personally I think this is a horrible idea. Gauging support is one of the BIGGEST parts of moving submissions from discussion to the actual poll. There were parts of CAP 5 where I had to go on judgement alone along with support from users. Another huge part of things like movepool (and really any other submission poll) is that ALL submitters have the ability to respond to feedback, whether it's on IRC, the Server, or on the forums. That wouldn't be allowed here because you're limiting the democratic process in a sense. In congress, representatives don't just blindly submit bills to the speaker, and vote on them. They have a chance to defend their proposition and make changes accordingly. With this, there is no room for that crucial debate portion, except by proxy.

Then have the TL post the submissions in the thread, and anonymously support them. You don't have to have submitted to be able to support a submission. If you want feedback, then bring it up. I have absolutely no problem with claiming subA is yours, the only reason for this PR is to prevent 1-second voting based on name recognition.

Another reason why I don't support this proposal is because it puts a ton of strain on the TL. TLs already take enough crap from everyone when they don't put so and so's submission in. Some TLs could feasibly rig the poll, and nobody would know the difference if they can't reveal the submitter!

I agree that it makes more work for the TL, however we've never had reason or work for a aTL. There are numerous people able to take the aTL position. The ability for a TL to rig a poll has not changed. Any TL could easily remove a very viable choice from the poll with or without this change. If there is a real problem with this, I'm sure it could be dealt with.

 

[darkie]

And also, I would like that if we were to continue this argument, Darkie, that we do so via PM. We have already clogged up this PR thread enough as it is.

It's not an argument - or at least I'm not arguing. It's a discussion. Discussion happens in PR threads.

I guess you misunderstood what I was trying to get across. I am saying that in Situation 1 that you Can't look at the submissions, not that our voter is too lazy. Knowing the more famous members of the community, and without being able to look at the actual submission, this intelligent voter votes logically for X-Act, as he has made and is still making many intelligent, useful resources and such for the community. Kaxtar, on the other hand, he/she has never herd of before. So he/she comes to the conclusion of "I'd rather vote for a genuis than a noob", and votes for the genuis because he can't see what he is voting for.

These two situations are different, because it can help to understand the effects of one situation if you can start by looking by their oppisite. So they are not exactly comparable, but by looking at one you can help to see the other more clearly, and that's exactly why Situation 1 is there. Now Situation 2 is vastly different, because one is the reverse of the other. Now the situation is that "if I can only see what I'm voting for, but not who posted it, I should vote for the one that appeals to me more." So in this hypothetical case, he ends up voting for Kaxtar over X-Act because, for some reason, he likes mine over X-Act's.

Okay. When will a situation where a voter can see the names but not the submissions itself ever happen? The only reasons I can come up with are that the TL doesn't add them (which won't happen so long as TLs do what TLs are supposed to do), a browser malfunction (rare), or said voter's internet cutting off in the middle of loading the page (in which case their vote wouldn't go through anyway.) I don't understand why you're thinking of situations which will never happen.

 

[Kaxtar]

Okay. When will a situation where a voter can see the names but not the submissions itself ever happen? The only reasons I can come up with are that the TL doesn't add them (which won't happen so long as TLs do what TLs are supposed to do), a browser malfunction (rare), or said voter's internet cutting off in the middle of loading the page (in which case their vote wouldn't go through anyway.) I don't understand why you're thinking of situations which will never happen.

All right, I suppose modifying Situation 1 could be helpful. I did, however, never think that Situation 1 could happen.

All right, our voter can look at submissions, but instead chooses to only look at the submissions of those who are well respected. Or that our voter will vote for famous person X right from the get-go. Either way, many submissions have been automatically skipped. There should not be a majority of voters skipping a majority of submissions, no matter who posetd that submission. It gives more people a better chance to win polls if most polls are done blind. Things should be a fair as possible, and if fame is out of the way, things become all the more fair. If things could always be similar to Situation 2, things could end up like this

If I can only see what I'm voting for, but not who posted it, I should vote for the one that appeals to me more

In Situation 2, everyone has to vote for the one they like more. If we could eliminate bandwagoning based on popularity alone, we should do it. So everyone gets what they really like when voting. For all you know, you could be voting for a submission you don't really want if you vote based on names. And if any person wins any poll without fame involved, bragging rights become all the better.

 

[Grimoire]

In Situation 2, everyone has to vote for the one they like more.

Or they would vote based on number. I have met a scary number of people who will vote for number 7, because 7 is a lucky number. To use your hypothetical. If all you had were numbers and weren't allowed to see the submissions which would you vote for? As stated I know many people who would vote 7. I also know some who have an uncanny liking for the number five. Me? I probably wouldn't vote(just as if I could only see the names), but if someone strong-armed me into voting I would probably go for three(just like if I was strong-armed into voting in your situation 1 I would carefully look over all the listed peoples previous entries/posts and use that as an analysis for voting. Many people tend to work in patterns after all).



Anyways I just don't see blind polling doing anything significant other than creating more work.

 

[Magmortified]

It's not so much a bad thing as it is incredibly convoluted. Are we even doing anything by this proposal? Who can tell? Does the thing this pr thread is attempting to address even exist? Who can tell? Are we going to have to do more work and remove personality from CAP because of something we're not even sure of? Hell yes.

Either way, many submissions have been automatically skipped.

Who are you to say this? Where is your information coming from? For what reason can you claim something so large without any seeming information?

It gives more people a better chance to win polls if most polls are done blind.

Are we even sure if people are voting based on name alone to begin with? There's been very, very little evidence that'd actually support this. Are we prepared to institute a policy that would only hinder CAP's inclusive atmosphere, and only compound work for the people overseeing the project... for what could only be minimal benefit?

And, seriously, there'd be no way to identify if we're doing anything right because we've got absolutely nothing to objectively compare it to. We're not sure if it'll help, and we're not sure if it's helping when we're doing it. Meanwhile, it's making other peoples' jobs more difficult, and making CAP seem paranoid.

Things should be a fair as possible, and if fame is out of the way, things become all the more fair.

How much more fair for how much more work and, let's face it, lack of personality? It seems so impersonal when you're referring to everything by numbers. "Hello, no. 242!" "Hey, Bob!" Honestly, which one seems friendlier and more inviting?

Why are we shooting at shadows?

If things could always be similar to Situation 2, things could end up like this

If, if, if. What about "will"? Let's face it: this entire thing is built on something with extremely little evidence. I respect Deck's contributions, but, let's face it: doesn't this:

foibles later employed the “I was kidding” defense or some such.

Seem a little paranoid to you? In the context I read that in, it seemed very much like a joke. It was late in the project, late in the poll... if there was a time for a joke, it'd have been there.

If we could eliminate bandwagoning based on popularity alone, we should do it.

Or we could chase other boogeymen that we only have the barest shred of evidence as to their existence.

So everyone gets what they really like when voting. For all you know, you could be voting for a submission you don't really want if you vote based on names.

That's why people do tend to, y'know, read the thing.

And if any person wins any poll without fame involved, bragging rights become all the better.

We won't ever know if fame was involved either way. ._.

 

[cyberzero]

Although the proposition is good in theory, I must disagree because of the sole reason that there is no way to gain constructive criticism from such a process because every minor change would have to go through the TL. If we can't have someone post and have his contribution reviewed by the community, it can't be the best that it can be and therefore directly taking away the quality of the CAP.

 

[ferron]

kaxtar, statistical analysis and psychology dictate that; people are less likely to vote in a blind poll or study unless previously agreed to be in said poll or study, people are less likely to vote if they feel as though it would be a hassel to do so. check your stat books on the section for possible errors, and your psyche books on the statistically analysis section (usually chapter 2~3 or a appendix in the back for highschool and college psyche books).
the reason i bring up the first one is that none of us are contracted to vote, so thus we can not force people to vote in a poll. the fact that name recognition is part of the polls actually will mean that people who vote to see "what ______ has turned in this time" will not bother to vote at all. note that this person is merely checking out the submission and not necessary voting for it (though there is a skewwed chance that they will).
the second one i brought up is due to the idea of PM voting that people have brought up (and thus not really targeted at kaxtar). PM voting only takes about 5 seconds more than bold voting in a thread, but it FEELS like more of a hassel. this will have the possibility to lower the number of votes.
...........
as for worries of submissions being skipped over, the best way to do that would be to break up the voting into 4 to 6 submission groups that all run at the same time. then taking the top 2~3 from each group, you form new random groups and go again.
the lower number of submissions per poll makes it more likely for people to read each one, and the randomization of the following groups makes the chance of voting by name less likely.
the problem with this is that it is likely to triple the amount of time that is needed to do a stat or movepool poll, thus making it less useful for us. knowing this is still bring it up because we need alternatives to blind polls to be brought up.

 

[darkie]

I don't think making an already convoluted process even more complicated is the solution, ferron.

 

[ferron]

I don't think making an already convoluted process even more complicated is the solution, ferron.

the problem with this is that it is likely to triple the amount of time that is needed to do a stat or movepool poll, thus making it less useful for us. knowing this is still bring it up because we need alternatives to blind polls to be brought up.

i'm not saying its a good solution, but we need more options than just "keep it the same" and "do blind polls". i am trying to think outside the box to get some discussion going for other possibilities.

 

[billymills]

The way this thread is going is really counter-productive. It's clear there's many people who do not want blind polling.

As an in-between, I think we should refer to all submissions without usernames. For example, for GHOSTEEL concept we were voting on 'Beej's Scouter' the addition of the submitter's name does nothing, and it can only add bias. There is no point to it. Just say Scouter, if there are multiple similar ideas, we should attempt to combine the submissions.

In addition, basestat spreads should be neutral as well. ie: submissionA, submissionB, etc. It does not matter if you mention names in every single post, just do not mention them in the OP and poll. If people want to take the time to read through the whole submission thread to figure out whos is whos, let them. The OP and poll should however, remain completely unbiased.

For anyone who is too naïve to believe that bias does not exist, it does.

 

[User 12783]

One question: How would you delete that bias when you quote the submission? The name of the creator will appear either way, so yeah...

 

[darkie]

One question: How would you delete that bias when you quote the submission? The name of the creator will appear either way, so yeah...

Not exactly...

 

[Serenity]

There will be bias no matter how we put it. All this would do is make more work for the TL, who, chances are, with all this change will make more mistakes than usual then all the blame go to him. This kinda thing would be easily accomplished with bold voting where you would be able to change your vote, which was pointed out as unnecessary. Everything you said I tried to convey in my policy review prior to cap7.

 

[cyberzero]

Serenity, the bold voting caused many problems this time around because Mag miscounted a few of the changed votes. Bias is a part of CAP, and whether we like it or not, it's hard to avoid it completely. This proposal is a possible way of reducing bias, but it's not worth it in my opinion.

 

[billymills]

Serenity, the bold voting caused many problems this time around because Mag miscounted a few of the changed votes. Bias is a part of CAP, and whether we like it or not, it's hard to avoid it completely. This proposal is a possible way of reducing bias, but it's not worth it in my opinion.

Just because the TL made a ton of errors last time doesn't mean the system is flawed in anyway. If it's a possible way of helping, then why not try it? The worst that can come of it is that we have the same thing that happened with this one, but we learn that this new process doesn't work great.

 

[Deck Knight]

Just because the TL made a ton of errors last time doesn't mean the system is flawed in anyway. If it's a possible way of helping, then why not try it? The worst that can come of it is that we have the same thing that happened with this one, but we learn that this new process doesn't work great.

Mag made one mistake in the first counting and it didn't make much difference. I'll also admit the New Move Submission Thread could have been improved. That being said, if Theoretical Mag gets submissions in and they conform to what the polls have outlined, I doubt he's going to leave one out because "user tennis whines too much. User Deck Knight causes too much trouble. I don't like users latinoheat or cyberzero etc. etc."

The arguments regarding "rigging" are absolutely ridiculous. Unless we nominate a random Firebot-only poster we aren't going to have any issues with that, because the blind process submissions occur at the point with the lowest stakes.

If we're really worried about reduced votes, does it actually matter whether we get 200 voters on a poll or only 120? If people do not have a real interest in this project and can't bring themselves to read the topic and make an informed decision, then I don't want them voting. Period.

Major competitive polls can be decided by 5 votes going differently. Any bias, no matter how small, can have an effect in such a close contest. If we have a Submission Summary at the end of the poll with the names of all submitters over the course of the polls, we're basically asking for delayed gratification.

In any event, I know the process itself is more difficult to implement, especially as regarding the early stages of base stats submissions. That being said, if you're serious about submitting, you can keep working on your spread until the Bias and BSR polls are all completed. Before anything is submitted you can basically discussn where you want it to go. Granted if you're all adamant about making it mixed and yours is the only spread with say, 70 SA while everyone else has 55, that might "out" you, but only if you were actually cognizant of the discussion thread, in which case you'd be making an informed decision anyway.

 

[DougJustDoug]

Conclusion

The proposal will not be implemented in its current form. If a more workable proposal is made, we can reconsider it.


Explanation

This was a difficult proposal to conclude. I think there is certainly split opinions on whether it was necessary at all. There seemed to be a general consensus that art and sprite threads should be exempted from this. Normally, I would hold a PRC vote on the part where there was split opinion. However, I discussed this with Bass, and tried figure out exactly what to include in the OP of a voting thread. In doing so, I realized that this proposal isn't really workable in its current form.

If we implement a general rule prohibiting the identification of submissions, what other rules do we need to make?

• Do we also make a rule prohibiting people from identifying their submission in thread posts?
• Do we also make a rule prohibiting people from identifying other people's submission in thread posts?
• Do we also make a rule prohibiting people from implying or speculating on the identity of a submitter?
• Can you identify your submission after it has been eliminated? (Probably not, since it narrows the possible identities of the remaining options)
• Can you identify that you are a submitter at all?
• What if people identify submissions in the CAP server mainchat?
• What is the penalty for breaking any of the rules above? Is it an infraction? Are you disqualified?
• etc.. etc.. etc...

If we implement the general rule, I really don't want to leave all this other stuff up to "mod discretion" or "TL discretion". Unfortunately, we would need to spell out all the other rules, or at least give some guidelines. The CAP process already has a ton of rules and guidelines, I'm not keen on making a general rule that turns out to be a headache for the mods, and possibly is unenforceable.

I am not saying it is impossible to make a workable rule for this. But, I don't think a workable rule has been defined in this PR thread. Therefore, we will not proceed with this exact proposal. If someone wants to try and make a proposal along similar lines, that is fairly straightforward and clear -- we can consider it in a later Policy Review.