12

On the most recent photo competition, All Blur, there has been a comment:

Has anyone else noticed that the "winners" of these are always among the first half dozen "answer" posts? Perhaps we need to ask people not to vote for a few days to allow more equal voting...

Then there is a proposed solution:

Perhaps an open for submission time period, then closed. Then an open for voting time period.

It has also been mentioned in the comments that the above effect happens on all SE sites. However, since the photo competitions generate many answers (the most recent competitions had 35 and 56 answers), I would assume that it is more extreme here.

Would a solution such as the one proposed above be favoured by this community? Is there anything else that we could do to make the voting more fair?

8
  • 1
    I don't think there's any convenient way. The only way to prevent voting is by locking each answer, which will also prevent any interactions (comment, edit), and can only be done by moderators. The alternative is to post an answer then delete it as soon as possible to prevent voting, then perhaps all of them can be undeleted by moderators at the same time when the voting time begins. Otherwise, the proposed solution cannot be enforced since anyone with the privilege can still upvote as they like.
    – Andrew T.
    Apr 13 at 15:03
  • 1
    ... Though, preventing early upvoting will also reduce the hotness of the competition thread, and might not make it enter the HNQ (that is broadcasted on almost all SE sites), which might also reduce participation, if that's the intended side-effect...
    – Andrew T.
    Apr 13 at 15:10
  • 4
    The issue is that most people sort by votes and that most are only going to look at 5 or so images. So it pops in HNQ, people upvote the first couple they see and move on. A potential fix would be to display the submissions in a random order and mask the vote count till the end which I guess is another way of saying the problem is SE not being a very good setup for this kind of thing.
    – eps
    Apr 13 at 20:01
  • 2
    I've tagged this as featured to draw attention to it on the main site (in the yellow sidebar for desktop viewers) (bottom of the page for mobile viewers.... yeah, probably not that visible for them)
    – scottbb Mod
    Apr 14 at 0:10
  • 1
    The photo contests will always be problematic because they don't align with the StackExchange format. Even deeper, the photo contest promotes a very narrow idea about photography...the single heroic image. And even worse, it's inherent competitiveness sets ground rules for the site due to its official sanction. I mean this question is motivated by "unhappiness" with the contests. Unsurprising because they are literally popularity contests. Apr 19 at 6:10
  • As this entire page seems to be devoted to rewarding lack of effort, I'll leave those 'motivated by unhappiness' to it. I will no longer be participating in the competitions.
    – Tetsujin
    Apr 26 at 14:32
  • 2
    Imagine how horrible the stack exchange network would be if it were based on the idea of competition to lead to a consensus of what the best/correct answers are! Oh... wait...
    – Michael C
    Apr 28 at 2:01
  • Like I thought. Contests (even here) are a waste of time and effort, and reward something I don't even know how to define, never mind excel at.
    – Zeiss Ikon
    Apr 29 at 19:23
5

Maybe to stimulate viewers/voters to not sort by Votes, but by Active some text can be added to the "Voting rules" section?

The text could either suggest that viewers sort by Active, or actually include a link to the Active sorting view, i.e. with the URL parameter ?answertab=active#tab-top.
For the 2021-04-26 competition that would be:

Photo Competition 2021-04-26: All Blur


For example here is my suggestion for the "Voting rules" text:

Voting Rules


A "downside" I just noticed of directing users to the Active view via URL is that this setting (sort by Active) "sticks". So anyone viewing answers on Photo.SE after clicking the link will view all answers sorted by Active from then on. This seems not to carry over to other SE sites.

Of course this is easily reversed by choosing "Votes" as a sorting option the next time they view answers, but it might cause some irritation or confusion.

12
  • 1
    I really like this suggestion. I normally prefer to view Photo-SE questions sorted by "Active", and I didn't notice that at some point my default view had changed to "Votes". I'll have to look into the conditions / reasons that the modes "stick" or don't stick depending on the site. But this is promising.
    – scottbb Mod
    Apr 14 at 20:25
  • Wouldn't "Active" just bias to a different set, namely the newer ones and the ones that have just been voted on? Apr 16 at 20:11
  • 1
    Does voting on an answer make it "active"? Yes, it would create a bias towards newer submissions, but as people keep sending in submissions the set of photos that "enjoy" this bias changes, in contrast to sorting by votes, where a small set of early submissions keep receiving the votes. Apr 16 at 21:54
  • 2
    "Cool, that means all I have to do to win is edit my submission a couple times a day to keep it at the top." - That one really competitive person who REALLY wants to win. Apr 19 at 20:21
  • Thanks for the feedback @LightBender. That is indeed a possible issue. Do you maybe have a suggestion how that can be prevented? I think serial editing becomes obvious pretty fast, given that edit history is tracked. And regardless, it's not that I'm trying to suggest a world-class photo competition system, I'm just trying to make do with the limited tools available. Apr 19 at 21:27
  • I'm doing a little statistical analysis on the data from the last few competitions, trying to figure out what the actual effect is. So far I'm seeing 70-80% are submitted in the first 72 hours and at least one of those will be in the bottom 5. I wonder if the first few posters are just the ones who track the upcoming themes and as a result have days or weeks to shoot or find a great image. In the current competition, I saw the theme coming up and had time to dig up a picture I hadn't though about in years just because it fit the theme. How much does that preparation affect the rankings? Apr 19 at 21:41
  • I added the "View the submissions sorted by active..." rule to the April 26 contest. It'll be interesting to see if the voting patterns change substantially (ping @LightBender)
    – scottbb Mod
    Apr 26 at 2:34
  • @scottbb I hope this works, but it will be very hard to enforce. You might also add a rule that disallows editing of posts after the next image has been submitted to prevent the potential abuse pointed out above. Apr 26 at 2:50
  • @LightBender It's my hope that submitters are honest enough to not game the Active sort with editing. IMO incidental editing should be allowed (make corrections, etc.). But if we notice gamification, the mods have other levers that we can pull... ;-)
    – scottbb Mod
    Apr 26 at 2:53
  • @scottbb I hope so as well, and I do doubt any of our regular competitors would engage in such unsavory behavior. I think it far more likely that a perfectionist such as myself might repeatedly make corrections without realizing its potential impact on the voting. However, I trust your judgement and if it does not concern you, it doesn't concern me. Apr 26 at 3:03
  • @scottbb I'm curious if there'll be an effect. Also, in case you want to do this in the future, apparently you have to use the long link of the question (i.e. with the full title), otherwise the ?answertab=active#tab-top parameter does not seem to work (am using Brave on a Mac if that matters). I've edited the contest question to fix that. Apr 26 at 10:26
  • @SaaruLindestøkke thanks for catching that.
    – scottbb Mod
    Apr 26 at 13:09
8

After doing a little statistical analysis, time of the post appears have little effect in ranking until after the 48 hour mark of the competition. Then in compounds for every day later. The data set is limited but...

The top 3-5 tend to score significantly higher than other posts submitted before or between those posts often double or triple the point totals of the next highest ranked post. Posts from the first 24 hours are can easily drift down to the bottom quintile. Posts from the first 72 hours of the competition are over represented in the top quintile and under represented in the bottom; posts from the last 72 hours do not hold the same pattern in reverse. Both are generally well represented in all three middle quintiles.

Roughly half of all submissions are made in the first 24 hours, and 70-80% are made in the first 72.

Basically, an image posted in the first few hours is about as likely to end up in the fourth quintile as the second.

I can't say with any conclusiveness, but I suspect the remaining discrepancy may be down to the preparedness of the first few submissions. In order to submit in the first hour or two, you'd almost have to be following the competition closely enough to know what the next theme is going to be, which gives you an advantage just because you have more time to consider your submission.

As a personal example, I've been doing a lot of street photography since I moved to the NYC area 5 years ago. Since I saw that street photography was an upcoming category, somewhere in the back of my mind a part of me has been internally debating which image(s) from the hundreds in my catalog I might submit*. To my mind, that gives a competitor a distinct advantage over someone who discovered the contest theme when the contest was posted.

So as to a recommendation, this should reduce the impact and not drastically increase the work for Scott:

  • Display the upcoming theme under the featured image about a week before it opens, "Upcoming Contest - Theme - Starts YYYY-MM-DD".
  • Those interested in participating can check into the site any time after it is announced, giving them time to consider and warning them when the contest will open so they can get their submission in early.
  • State the known issue in the rules and actively encourage competitors to submit their images on the first day of the contest if possible.

*And which I don't mind falling under the Creative Commons license, which is a far bigger consideration for me

9
  • 2
    You had me at "a little statistical analysis." Apr 21 at 11:51
  • "Post the contest, locked, a week before it opens, "Upcoming Contest - Theme - Starts YYYY-MM-DD".' This would be great.
    – Eric S
    Apr 21 at 16:31
  • It would be awesome to see some more detail about the statistical analysis. Apr 22 at 16:13
  • Locking the post for a week will essentially shot down the chance to enter HNQ (and the overall exposure to the network and total votes), if it's something to be considered by the community.
    – Andrew T.
    Apr 23 at 2:49
  • I may be incorrect here, but qualifying for HNQ is about recent interest and activity (votes and answers) on a question, not how old a question is. The vast majority of HNQ are very recent, but you do see older posts gather sudden traction and make it into the list. If I am mistaken, this is not a barrier to the overall fix as there are other ways to announce the upcoming theme several days in advance, most likely under the photo of the week description. Apr 23 at 12:55
  • There's a minimum age of 8-hour for questions to be eligible to enter HNQ, but the age of the question is also a degrading factor (reference: the answer to What are the criteria for questions to be selected for Hot Network Questions?). Personally, I've never seen a question that has past 3 days to enter HNQ though...
    – Andrew T.
    Apr 24 at 2:49
  • @AndrewT. Thank you for that information, I will update my answer to suggest putting the next theme under the featured image instead. Apr 24 at 2:51
  • 1
    @BobMacaroniMcStevens It was a simple probability analysis mapping the actual intake time in 24 hour increments to the actual final ranking (divided into quintiles due to the limited data available) and comparing that to the same images in a hypothetical normalized block where 50% of the remaining votes are randomly cast on the submitted images every 48 hour throughout the competition. (Based on the competition usually hitting the HNQ list on day 1 and remaining there throughout day 2.) Then comparing the random quintile spread to the actual results for each day. Apr 24 at 3:02
  • I forgot to mention, re: "Display the upcoming theme under the featured image about a week before it opens"... can't be done right now. I can only fill in 4 fields in the PotW sidebar: 1) Photo title; 2) Photo imgur URL; 3) User account URL; 4) Contest submission URL in Meta-PSE.
    – scottbb Mod
    Apr 27 at 4:30
5

I would just like a months warning of the topic to look through my archives and prepare a submission. This way most pictures will get posted closer in time.

1
3

Here's an obvious solution to this problem, but it would require work at the top level. All SE sites could benefit from it.

Provide individual questions with, in addition to Active, Oldest and Votes, a Random setting that causes answers to display in random order. The owner of the question could set its default value.

If you and I look at the contest question now, and then again later, the answers will be presented to us in four totally different orders.

The snowballing compound interest effect that we see now would be eliminated.

UPDATE:
Based on scottbb♦'s comment, rather than allowing the owner of the question to set the initial random flag, the shuffling should be triggered by a moderator-only settable tag, say (ontological) or (teleological).

3
  • Hmm. Other than the photo competition, I'm not sure having question owners being able to dictate users' view order is generally beneficial. Are there other cases in this or other Stacks where you think this might be beneficial (besides contest-style questions, which are probably an abuse of the SE Q&A model anyways)?
    – scottbb Mod
    Apr 15 at 14:50
  • It wouldn't be permanently forcing the random ordering. That would be only the default for these kinds of questions; users could still manually select any of the other orderings. (And when the contest is over, "Votes" would be set as the default ordering.) Apr 16 at 0:27
  • 2
    It doesn't completely address your described use-case, but there is a Real random ordering of answers feature request at Meta Stack Exchange. Of course, it's still per-viewer preference, not controlled at all by the question asker (or perhaps triggered by a particular site keyword/tag), but the random sort is a currently-open feature request.
    – scottbb Mod
    Apr 16 at 21:35
3

Get rid of selection by voting altogether and pick a picture at random from the submissions. People can still share their work. People can say nice things about each other's pictures in the comments.

Edit 1: I think there might be existing tools on the StackExchange platform for random selection of ordinary content used for choosing items to display in places like site front pages and the sidebars. If so, this might allow the moderators to automate additional aspects of the work.

Edit 2: The general concept could be more of a group exhibition rather than a competition. This might create opportunities for several people to create pictures that respond to and resonate with each other.

16
  • This is probably the most practical suggestion since it will work similar to how community ads work.
    – Andrew T.
    Apr 21 at 10:07
  • 2
    So your solution to do away with competition all together? And to "say nice things." What benefit is that to a community who's primary goal is to help others learn and grow. If it makes no difference if I submit a good photo or a bad photo, what do I learn? If there are 50 entrants, 49 of them are going to lose, and 49 of them are going to be unhappy that they lost. This is not a bad thing because they can learn from how they ranked. A random selection removes this benefit and leaves you with 49 losers who lost for no reason. Apr 21 at 12:28
  • @LightBender I am glad we agree that the primary result of the photo contest is unhappiness. Apr 21 at 13:31
  • 2
    As it is for all competition worth engaging in. The OP's question is how can we reduce the impact of having a competition within the confines of the SO model. If the field of play is not level, we should level it if we can, but there is no savor in a win that was handed to you without regard to performance. I won't presume to speculate why you've never submitted an image, but I see no reason to deny competition to those who find it valuable just because you seem not to. Apr 21 at 15:45
  • 1
    I do not remember agreeing that there is little to savor. Quite the opposite. I wholeheartedly reject your premise. What you see as the evils of the competition, I see as the competition's greatest benefits. The apparent tendency to misrepresenting another's options, intentionally or unintentionally, to force them to agree with you is most illuminating though. Apr 21 at 18:20
  • @LightBender my apologies for misrepresenting your comment. Apr 21 at 18:40
  • @LightBender I don’t enter because the primary product is unhappiness. Apr 21 at 18:50
  • 1
    @LightBender " they can learn from how they ranked." - what exactly do you "learn" by getting a different number of votes than someone else's post? Nothing at all, unless some of the voters also give some feedback!
    – alephzero
    Apr 23 at 18:44
  • @alephzero You can learn a great deal in fact. On its own, how your work is received generally. In concert with other answers, you can compare your work to high ranking images and glean insight into what those images share that resonates with others, discover things that you share with lower ranking images to determine what aspects you might be distracting from your intent. While you might not pick up special pointers about your technical performance, there is a wealth of information to be learned about the "soft" skills that separate the merely competent from the true artist. Apr 24 at 2:20
  • A key skill for any artist is to create art that communicates with people. Be it a specific client, a community or the public at large, the soft skill of being able to understand your audience and speak directly to them is invaluable. Just as I wouldn't explain something to my 7 year old niece as I would one of my coworkers, every image can be made more effective by the careful consideration of the audience. Practicing this deliberately and often with a wide variety of potential audiences improves this skill immeasurably over time. No explicit critique necessary. Apr 24 at 2:30
  • @LightBender Have you some formal education experience in a creative studio discipline? Apr 24 at 18:18
  • I do in fact, formal training in how to teach as well as both learning and teaching creative disciplines in a formal setting for over 20 years. Since we've gotten to a form of the call to authority fallacy, I will now consider the conversation over. Apr 25 at 1:02
  • @LightBender Was winner takes all competition a primary pedagogical tool in those settings? I asked the question to understand where you are coming from. Apr 25 at 3:41
  • @LightBender Judging opinions based on relevant experience and expertise is non fallacious. The argument from authority fallacy consists of appealing to an external source whose experience is not relevant to the matter at hand. Apr 25 at 3:47
  • 3
    I've never won, but I've never been "unhappy" with the results, nor have I been "unhappy" with how my entrants did.
    – Michael C
    Apr 28 at 1:33
2

One way to mitigate this effect would be to have two separate questions: a call for submissions, and then a separate question for voting.

The idea would be that the competitors would post their photos as answers to the "call for submissions" question. Then, after the submission deadline, the person running the competition would copy all the answers to the "judging the competition" question and set them to community wiki. Then people would vote on these copies of the answers, which would be fairer since they'd all appear on the same day.

This isn't perfect though. It solves the problem of the earlier answers having a longer time to accumulate votes, but it doesn't solve the problem that for most people the highest-voted answer will be at the top, so there will tend to be a snowball effect. It also makes extra work for the person running the competition. It could be worth considering though.

4
  • 1
    Two issues: 1. as the person running the competition, that's a lot of extra work. Especially in cases where there are 20+ submissions. There's no ability for me (or another mod) to keep the "judging" question closed / locked while each answer is being copied over. The only way I can do that is to unlock the question so I can click on "Answer this question", then quickly lock the question in another tab, copy the answer over, then unlock the question long enough to finalize adding the copied answer, click "Answer this question", lock it, etc... This is really abusing the Q&A system, as I see it
    – scottbb Mod
    Apr 16 at 21:22
  • 1
    I wouldn't want to set a precedent to another mod if I'm gone or I leave, for them to have to do that work, also. #2: There's no way for mods to create answers in another user's name. So the "judging" question's answers will all be in the mod's name. And as far as I know, there's no way to prevent additional answers from being posted as submissions, so there's more cleanup / monitoring involved, for what would otherwise appear to be a legitimate way for new people to answer/respond. I think that creates potential confusion on the parts of some brand new users/participants.
    – scottbb Mod
    Apr 16 at 21:25
  • 1
    However, I do like the suggestion about community wiki for the contest. I hadn't considered that before. I'm going to dig around and ask about that some more.
    – scottbb Mod
    Apr 16 at 21:26
  • 2
    I made the April 26 2021 contest CW. I wonder if that will have any effect on submissions (hopefully the lack of rep won't discourage submitters)
    – scottbb Mod
    Apr 26 at 2:36
2

As was suggested by Andrew, in a comment to Bob Macaroni McStevens answer:

"This is probably the most practical suggestion since it will work similar to how community ads work. – Andrew T. 2 days ago"

We could use an existing mechanism, that solves our problems without a lot of extra work, is useful for more than one site, and doesn't create an unwanted legacy of extra work for future moderators:

" ... It solves the problem of the earlier answers having a longer time to accumulate votes, but it doesn't solve the problem that for most people the highest-voted answer will be at the top, so there will tend to be a snowball effect. It also makes extra work for the person running the competition." - in an answer from Nathaniel.

No new coding for randomization, or precision manually locking and unlocking. We make a request on MSE for a moderator controlled bit (much like a community wiki) for contests (which are also popular on that site) using an existing mechanism; with minimal editing to the wording and minimal use of expensive developer time.

Use the existing Election code - the features we need, with built-in fairness.

For the third stage we don't necessarily need to use the audited third party election tally site (OpaVote), we could simply rely on votes. This method allows a discovery and feedback phase, a means to reduce a huge number of entries to a smaller pick, and a final voting phase.

The top banner and question area text would be editable by a moderator, rather than a community manager. Contest winners could retire to a new webpage, similar to the past and current moderators list, complete with links to the prior contests; possibly with page turning links if there's going to be a dozen contests per year.

3
1

One idea is to convert the contest into a locked question with one community wiki "answer" with a strict formatting guideline. People would either directly edit the community wiki answer to add their photos, or if they do not have enough rep, they would add their answer to another question post. Any user, or moderator, can move these answers into the community wiki page, and the original answer would be deleted. In the last competition, everyone had enough rep to add their answer directly, so hopefully this won't be too much work and will just accommodate a few low-rep people. Alternatively, we just stick to having only >100 rep users participate and not use the separate submission system.

After the entry period of the competition is over, the community wiki post is locked, and the voting takes place on one of the multitude of online voting/polling/survey websites (eg. google forms) that will be linked in the question and community wiki post. The winners from that vote would get their photos featured.

4
  • "Alternatively, we just stick to having only >100 rep users participate." perhaps you meant protected question. However, it's not "only >100 rep users", but "users gaining 10 rep on this site", meaning: 10+ rep (new users) or 110+ rep (users with association bonus).
    – Andrew T.
    Apr 23 at 3:02
  • Oh I'm sorry if that was unclear. What I meant was if we stuck to just the 1 community wiki post then only 100 rep or greater users would be able to edit it and participate. Though I suppose they could still suggest edits. Apr 23 at 5:14
  • re: your last paragraph: I'm not a fan of taking anything user-identity–related off-site, at least from a moderator-sponsored/asked viewpoint. Users can interact with each other however they wish, including exchanging emails, online profiles, etc. I don't recall a specific policy regarding mods communicating with users off-site for anything related to the site, but something feels... not quite right to me. I really think everything site-related needs to stay here, from a mod perspective.
    – scottbb Mod
    Apr 26 at 2:43
  • 1
    (clarification: mods absolutely should not communicate with users off-site for anything that can remotely be considered behavior issues, responding to complaints, etc.)
    – scottbb Mod
    Apr 26 at 2:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .