Should Online Forums Implement a Member Ranking System?

[kichigai]

[SOLVED] A Member Ranking System?

I notice that there are a lot of different levels of people here. Some answers are way over my head. In another forum I saw they were using some kind of ranking system that helps others to know how much or how little he or she knows.

Can we have that here? Something like High School, University, PhD, Teacher, Expert, Super-Expert or maybe just Grades or Levels.
I sometimes get answers that I'm not sure are reasonable or not cause I don't know anything about the guy or gal giving me an answer.

Maybe you could add more quizzes and use it as a way to make a ranking for members?

As you can guess, I got a long way to go, so my rank would be not so great, but I'm here to learn from you guys if I can.

 

[anti_crank]

It's impossible to do this reliably, since one cannot prevent members from getting a more knowledgeable friend to answer for them. As far as degrees or grades go, I don't suppose you've given any thought to the problem of validating such things. PF has thousands of members and only a handful of admins who have their hands full. It might not even have a mailing address for transcripts to be sent.
The mentors can be trusted to know their material on the math and science forums. Other than that, there's no way around making your own judgement call based on the coherence, plausibility and style of argument of individual posters.

 

[Greg Bernhardt]

kichigai, you have an interesting idea that I will think about see if there is a way to meet your request. In the meantime anti_crank is right, you can trust mentors and admins 100%. In time as you discuss and meet more members you will be able to recognize who is more reliable.

 

[what_are_electrons]

It's impossible to do this reliably, since one cannot prevent members from getting a more knowledgeable friend to answer for them. As far as degrees or grades go, I don't suppose you've given any thought to the problem of validating such things. PF has thousands of members and only a handful of admins who have their hands full. It might not even have a mailing address for transcripts to be sent.
The mentors can be trusted to know their material on the math and science forums. Other than that, there's no way around making your own judgement call based on the coherence, plausibility and style of argument of individual posters.

Kichigai suggested that ranking could be based on the PF quizzes and that sounds useful. As you wrote, it's not perfect, but it would help those who are still learning. I see no need to validate degrees. I think he/she was just giving a label, not saying our busy mentors should track evaluate every members for their level of education.

Yes, the mentors have a lot of advanced physics knowledge and they can help us, but they are not the only ones responding which is why it's called a forum and not a class. Kichigai was probably not doubting the mentors. He/she seemed to be doubting some of the other members who provide answers that would hopefully be correct, but might have some errors of some sort that would screw up other knowledge or cause some concepts to be wrongly understood.

I agree that each of us, who are older and a bit wiser about taking info with a grain of salt, can fend for ourselves but there are younger folks and college folks and late bloomers who may not be able to sort the BS from the good stuff.

I wonder if the mentors and Greg have ever considered this style and whether or not they see any real problem with asking members to take a set of quizzes and to accept some sort of label. For those who do not take any quizzes to establish a rank, they can be labeled as "Wildcard" or "Unverified Source" or something like that.

I say we give Kichigai's idea a bit more thought.

 

[Greg Bernhardt]

I wonder if the mentors and Greg have ever considered this style and whether or not they see any real problem with asking members to take a set of quizzes and to accept some sort of label. For those who do not take any quizzes to establish a rank, they can be labeled as "Wildcard" or "Unverified Source" or something like that.

I say we give Kichigai's idea a bit more thought.

Absolutely, this will be discussed with the rest of the staff and I'll get back to this topic when I formulate an update.

 

[enigma]

you can trust mentors and admins 100%.

Well, you can at least trust us to make a best effort. Honest mistakes can happen. You can also trust us to acknowledge when we have made a mistake.

 

[jimmy p]

I am not a number! I'm a FREE MAN!

 

[arildno]

I think one of the main advantages about the active mentor/admin line pursued at PF, is that people knows this, and hence, is a bit more careful with what they choose to post, in order not to incur their wrath
(over-eager replies have occurred though, :blush:, )

 

[Werdatothewise]

Kichigai suggested that ranking could be based on the PF quizzes and that sounds useful. As you wrote, it's not perfect, but it would help those who are still learning. I see no need to validate degrees. I think he/she was just giving a label, not saying our busy mentors should track evaluate every members for their level of education.

Yes, the mentors have a lot of advanced physics knowledge and they can help us, but they are not the only ones responding which is why it's called a forum and not a class. Kichigai was probably not doubting the mentors. He/she seemed to be doubting some of the other members who provide answers that would hopefully be correct, but might have some errors of some sort that would screw up other knowledge or cause some concepts to be wrongly understood.

I agree that each of us, who are older and a bit wiser about taking info with a grain of salt, can fend for ourselves but there are younger folks and college folks and late bloomers who may not be able to sort the BS from the good stuff.

I wonder if the mentors and Greg have ever considered this style and whether or not they see any real problem with asking members to take a set of quizzes and to accept some sort of label. For those who do not take any quizzes to establish a rank, they can be labeled as "Wildcard" or "Unverified Source" or something like that.

I say we give Kichigai's idea a bit more thought.

I really put a lot of thought into responding to this forum question. As a long time professor, I believe those who want to be "labeled" for a ranking should be a self choice decision. Those who wish not to take a set of quizzes to establish their degree rank should NOT be allowed to represent a mentor of degree. Perhaps the current "mentor group ~ and Greg" could use an auto .gif that would give more outward authentisity to the postings of those who have desired the icon of a "degreed" mentor? This is my opinion only. I am here to sap up what is being written, and to laugh with those who know that education comes also in the form of humor. ~Werdas'

 

[Greg Bernhardt]

Chroot and I have found a solution and it's in the works. We'll update you all soon.

 

[Werdatothewise]

Here you go ~ Auto .Gif

Greg and Company,

Auto .Gif

If this one doesn't work, let me know... I can find or make another...

~Werdas'

 

[chroot]

Please see the medal system we added here:

https://www.physicsforums.com/showthread.php?t=41166

This system should identify those members who the PF staff feel are knowledgeable enough to be considered trustworthy scientific authorities.

- Warren

 

[cronxeh]

Is an engineer considered a 'science expert' or you must have a degree in sciences?

 

[Greg Bernhardt]

Is an engineer considered a 'science expert' or you must have a degree in sciences?

Yes, but we will be giving our awards based on discussions we review, not on specific academic and professional experience members might have.

 

[arildno]

I'm honoured.
I feel obliged to put a few more details in my user profile to highlight my "area of competence".

 

[marlon]

I am all for some sort of ranking system for the members of this forum. At least for all those that want to participate in such a system.

I think the best way to assign some kind of degree, will be based upon answers and solutions posted in threads.


I see only one problem : who will be assigning these "degrees".?

regards
marlon

 

[LURCH]

Here's an unforseen benifit:

O.K., so now I've got this little medal under my name that calls me a "science expert", and it's quite encouraging and even a bit flattering. But it also makes me think, "Eek! Now people will not only be reading what I print, but they'll actually think it's likely to be correct!". This might make people more carefull about what they state in a reply, and more diligent about clearly denoting what they know with certainty from what they are geussing or what they personally think. This should really help maintain the high standard of credibility that has made this site so popular in the first place.

Hey Greg, how 'bout a little icon of a cracked pot to put under certain PFer's names?

jk

 

[Greg Bernhardt]

Hey Greg, how 'bout a little icon of a cracked pot to put under certain PFer's names?

jk

haha, I'd rather us try to phase them out instead

 

[chroot]

O.K., so now I've got this little medal under my name that calls me a "science expert", and it's quite encouraging and even a bit flattering.

Excellent! We were hoping to inspire these kinds of feelings. You medalists are a very important part of this forum, and we wanted to make sure you know it.

But it also makes me think, "Eek! Now people will not only be reading what I print, but they'll actually think it's likely to be correct!". This might make people more carefull about what they state in a reply, and more diligent about clearly denoting what they know with certainty from what they are geussing or what they personally think. This should really help maintain the high standard of credibility that has made this site so popular in the first place.

We certainly hope so -- we'd love to see people vying for medals.

- Warren

 

[Werdatothewise]

I'm impressed!

Excellent! We were hoping to inspire these kinds of feelings. You medalists are a very important part of this forum, and we wanted to make sure you know it.

We certainly hope so -- we'd love to see people vying for medals.

- Warren

Dear Warren and Greg, ...
I commend you for your unbias judgement in identifying those members who obtained their "medals."

Now, I won't have a problem recommending the PF to my students. As for me, at my young age <sic> I will join in the more academic pages as soon as I stop teaching. I also will fix up my profile to denote what capabilities I can contribute to the PF.

BUT, first and foremost, I want those in need to know I have my PH.D. in Psychology. Might come in handy

Part of the American myth is that people who are handed the skin of a dead sheep at graduating time think that it will keep their minds alive forever.
John mason Brown​

 

[daveed]

how do the mentors upgrade/people get acknowledged as mentors, anyways? for example, i believe selfadjoint was a mentor, and one day i looked and they were a super mentor? haha yup i don't know much about how the forum works -_-

 

[Moonbear]

I'm honoured.
I feel obliged to put a few more details in my user profile to highlight my "area of competence".

I did the same. Might be good to suggest this to all the medal honorees. I just listed my research areas of interest.

 

[enigma]

I'm honoured.
I feel obliged to put a few more details in my user profile to highlight my "area of competence".

A good idea. I agree with Moonbear that we should "persuade" the medalees to update their bios to indicate what they specialize in.

It would certainly solve the concern I brought up here

 

[NateTG]

Chiming in...

Thanks for the kudos. I was a bit surprised to see the medals show up next to my posts, and went looking for more info.

I think that enigma's concern about the medals lending undue credence to various posts is legitemate. Personally I have made posts, and started threads, regarding non-mainstream QM interpretation - where I am most certainly not an authority - because I wanted to see the discussion.

I would suggest that if you're providing recognition, a good medal to award would be 'constructive poster' for people who have good posting etiquette, and who provide good discussion or suggestions (even if they aren't necessarily particularly knowledgeable.)

 

[marlon]

I think a medal should only rewarded based upon the explanations of a member for questions posed by others...you need to help others out and doing that you remain "active" on a certain subject...

Ofcourse you need to be polite...

regards
marlon

 

[hemmul]

Hi all!

I am happy that this discussion started here :) this idea is the one that was tourturing me for a long time, but i just couldn't find a bunch of free minutes to write down and submit my opinions...
Now as the topic is started - let me add my 5 cents :)

You know, physicsforums.com is the one of the websites i visit happily nearly every day, after once i stumbled on it while searching some stuff in google... It was the only place with necessary information! Already in 5 minutes i was proud of being a member of this community! And I'm still proud of it! There are really a lot of fascinating discussions and a lot of interesting people from (according to the well-known poll!) all over the world.

One thing i noticed here - is that people don't hesitate to write! It's very nice to read a long posting carring on tons of useful, interesting and sometimes "weird" information. Many of them are already presented as a well-settled article :) and the majorty is invaluable!

But all of us are humans. Some posts are "good" some are "better" some are "brilliant" and some are "fantastisch!", and it doesn't depend on who is the author... In some particular cases, the simple answer posted by a "newbie" can be much more effective/creative than a strictly prooved theory of a "big guy" ;)

Why am i talking about this? Actually, sometimes, after/during a long, busy day i (and surely many of us) take a short tour to physicsforums just to "relax" a bit - just to read the interesting threads etc - whatever. But what i see is not a bunch of cool threads, no no... there are A LOT of GREAT threads! And there is no time to read ALL the answers in ALL the threads... And usually, while in a rush, i finish with a nimble global ALT+F4 for the opened threads :)

Introducing a rating system is an excellent idea! But as far as i understand You discuss here a member-rating system - medals etc... but what do You think of rating posts? Each message will have its own rating, which is cumulatively defined by the members rated it. Say there are ratings of -1, 1, 2, and 3, available. A perfect answer will be rated, say, by 12 members (the others haven't read the answer yet!). So the final score of a message will be 12*3=36. Simultaniously the same amount of points is added to its authors stack... Lately it is possible to use the authors rating to increase the points he gives the message while assessing it (some kind of "weight")

what are the advantages of such a system?
* It helps to select the most valuable and useful replies from those currently posted.
* It encourages people to help each other, and provides some kind of satisfaction - with no doubt, it is very nice when you see the feedback! Of course, the best award for one's help is a strong freindship with other interesting people, but putting a good mark is sometimes a best opportunity to thank the one who helped you!
* It discourages some people to write stupid, offensive etc posts (i never saw such ones in PF! - thanks mentors!) - who will like a tons of "-1"s accompaniyng their post?
* And also it works almast in the same way as the medals: members with more rating are more helpful, more popular etc... :)

disadvantages:
* the only disadvantage i see is that the rating of posts is subjective - some members may just forget to rate a post :) but, taking into the account the number of visitors of PF, such "errors" will be corrected by statistics!

Well, this is roughly what i think, based on the experience on other internet forums, and my own experience...
If somebody agrees, disagrees or just wants to discuss it i'll happily grant my help!

With best wishes to PF members,
Alex

 

[Moonbear]

Hemmul, we already have a thread rating system. It's just not used much. Look at the blue bar on the top of the first post on the page, just under the page numbers on the far right, it says, "rate this thread." Most people just ignore it though, so it doesn't help much. The only threads I've seen get 4 star ratings are the ones in general discussion, such as the member photo thread, that aren't about science at all, but are just fun.

 

[hemmul]

Hemmul, we already have a thread rating system. It's just not used much. Look at the blue bar on the top of the first post on the page, just under the page numbers on the far right, it says, "rate this thread." Most people just ignore it though, so it doesn't help much. The only threads I've seen get 4 star ratings are the ones in general discussion, such as the member photo thread, that aren't about science at all, but are just fun.

Thanks Moonbear for reply!
Actually i didn't mean thread rankings: i meant each post ranking! Of course the total thread ranking can be derived, say, as a sum of that for each post inside the thread...

Cheers,
Alex

 

[arildno]

hemmul:
IF enough members used this option, then I agree with you; it would be a nice addition.
However, based in part on the lack of response in the thread-ranking system, I think too few members would use it.
One reason why several members might choose not to use it, is the following:
Is an answer giving a clear, detailed solution better than a "dialogue of hints" leading up to an emerging understanding in the original poster?
That is, I think a lot of users will be unsure how to rate a given post.
(Other alternatives might be equally hard to choose the best from)

 

[Moonbear]

I agree with arildno on this. Since so few folks are even ranking threads, I doubt they'd take the time out to rank individual posts, so it would probably be a lot of effort for the admins to set up just to sit unused.

There are a lot of posts of the type arildno mentioned, ones where we choose not to give a complete answer so that someone trying to learn will learn to find the answers for themselves with just some prodding on our part.

I would also worry that the only people who would rank a post would be the crackpots who don't understand the difference between a scientific discussion and trying to win a debate at all costs and who dislike the experts who point out the cracks in their ideas. There seems to be a lot of room for abuse of such a feature.

And, I'm not really sure ranking individual posts would be all that useful. Often, it isn't a single post that makes a difference, but the sum of the posts contributing toward the discussion that makes a thread exciting.

 

[hemmul]

hemmul:
IF enough members used this option, then I agree with you; it would be a nice addition.
However, based in part on the lack of response in the thread-ranking system, I think too few members would use it.
One reason why several members might choose not to use it, is the following:
Is an answer giving a clear, detailed solution better than a "dialogue of hints" leading up to an emerging understanding in the original poster?
That is, I think a lot of users will be unsure how to rate a given post.
(Other alternatives might be equally hard to choose the best from)

Well it really has to be much more simpler: anyone can figure out whether he likes the post or not. if he likes it - he has to put a '3', if he dislikes it - put '-1' or put just nothing... the only problem, i agree, is to make people vote. They are really to be encouraged to :)

 

[Moonbear]

Well it really has to be much more simpler: anyone can figure out whether he likes the post or not. if he likes it - he has to put a '3', if he dislikes it - put '-1' or put just nothing... the only problem, i agree, is to make people vote. They are really to be encouraged to :)

That last part is the hardest! Probably if you can convince people to start using the thread ranking system, it might help at least raise the possibility that people would be also willing to try a post ranking feature.

 

[hemmul]

That last part is the hardest! Probably if you can convince people to start using the thread ranking system, it might help at least raise the possibility that people would be also willing to try a post ranking feature.

what about an adverticement service? say, "FREE AVATAR TO TO THOSE WHO GIVE THE FIRST 1000 RATINGS!" or "THE MEMBERS WHO RATE THE POST THEY READ, HAVE A CHANCE TO WIN $10 AT THE END OF EACH MONTH!"

well, at least we can launch a poll - at least it'd be clear who is the potential-mark-giver...

 

[arildno]

It's an interesting idea..

 

[Moonbear]

well, at least we can launch a poll - at least it'd be clear who is the potential-mark-giver...

Worth a try! Maybe posting a poll will serve to point out the feature exists for those who have never noticed the thread ranking option. Go for it!

 

[Gokul43201]

Consider this scenario : I tell a lazy high school kid that I won't do his homework for him, and that he'd better show some effort before expecting help. And he hates me for it and runs about giving all my posts a negative ranking ?

This could happen with school kids, pottery magicians, religious preachers...and it's the folks that do the right thing (not do others' homework for them; inform the crank that (s)he is potentially putting false ideas into the heads of people that come here to learn; show the preacher the way out) that are going to get poor ratings.

So, how do you decided what rating is valid and what isn't ?

Okay, you get the point.

 

[chroot]

We already have a "member ranking system" in the form of our awards. Rather than being automatic, they are chosen by hand. While the staff has a final say in who gets an award, anyone can nominate a member for an award. It's democratic, it's controlled, and so far, it seems to work quite well. I don't see the need for anything more complicated.

- Warren

 

[Gokul43201]

I nominate shmoe. I just noticed that he's not an Advisor !

 

[Math Is Hard]

I'd like to submit a nomination for chronos as a science advisor. Thanks.

 

[hemmul]

Consider this scenario : I tell a lazy high school kid that I won't do his homework for him, and that he'd better show some effort before expecting help. And he hates me for it and runs about giving all my posts a negative ranking ?

This could happen with school kids, pottery magicians, religious preachers...and it's the folks that do the right thing (not do others' homework for them; inform the crank that (s)he is potentially putting false ideas into the heads of people that come here to learn; show the preacher the way out) that are going to get poor ratings.

So, how do you decided what rating is valid and what isn't ?

Okay, you get the point.

Well, after chroot's "refuse" i don't think its a good idea for me to "struggle" more, but anyway i'll answer your question :)

There are several points. Firstly the rankings should be separate for each forum: if one is strong in physics it doesn't mean that he is as strong in biology (however it is not impossible) - so mixing up member-rankings across forums is not a good idea. The one who is strong in several fields of sciences will proove it!
So if someone rated your post negative in homework forum (actually imho homework foums do not need rankings at all) it will stay in homework forum. but that't not all. This cray guy is all alone. Others - who are grown up - understand that what you meant while not helping him was even more help than he expected. And they will rank your post positive because they agree with you. Now, seeing dozens of '+3' marks who cares about one single '-1'?
The only condition needed for this is that people vote... but according to the poll there are not much such people yet :) go! go! go!

Finally, one can remove the negative marks! So the worst thing one can do to you is not to rank your post at all or reply to your message and start discussion... but this is far away from the point...

 

[Clausius2]

For those who don't know me, I'm not a Ph.D and not a B.Sc. I'm only and student of a five years engineering graduate program. But I'm able to guess who is an expert on some subject and who isn't, in spite of the medals and rankings. I will trust in the answers of some people despites they have /have not any medal or label, simply because I have read enough posts of them to be sure.

And I'm not going to doubt of any answer of a subject which is not my main area of knowledge (which is reduced as you can observe ). The medals of Science Advisor, Homework Helper, or labels like PF Mentor, PF Supermentor, PF SuperSuperMentor, PF Requete-Mentor (spanish), are anulated automatically for me (and for you If I do something similar) if you say something without sense about some subject I know perfectly.

So that, I recommend the forum members to look not only at the medals, but through the medals.

 

[Gokul43201]

Clausius, the point of the medal is to help a newcomer decide between what is likely to be an accurate response and what isn't. If you think there are "Advisors" here that post incorrect information or responses, you should let Greg know. And, on the other hand, if you know of people whose responses you can trust for certain, you should suggest their names for a nomination.

Other's here might not have the advantage of knowing whose posts to believe, and that is the purpose of the medal. It is not meant primarily as a reward ! Or so I think.

 

[Clausius2]

Clausius, the point of the medal is to help a newcomer decide between what is likely to be an accurate response and what isn't. If you think there are "Advisors" here that post incorrect information or responses, you should let Greg know. And, on the other hand, if you know of people whose responses you can trust for certain, you should suggest their names for a nomination.

Other's here might not have the advantage of knowing whose posts to believe, and that is the purpose of the medal. It is not meant primarily as a reward ! Or so I think.

I don't know actually any advisor or Mentor who has given incorrect information, fortunately. It was only a general comment like: in the case of it happened...

I know the purpose of medals are well-intentioned. And I agree with you totally. But we are all human beings, and maybe to say something stupid some time. So that, an intelligent Forum member has to look into each answer before considering it correct merely because it's written by an Advisor or Mentor. But I say once again I agree with you and I'm aware of the good intention of such awards.

The fact that we have the capacity of enjudge any comment in books (also written by expert people) is what enhances our curiosity and proper ideas to learn more. This is a thing that has to be learned first of all by those who search for an answer to their problem in this forum.´

Hmmm. ... I think I'm going to leave engineering and become a philosopher...

 

[arildno]

But we are all human beings, and maybe to say something stupid some time. :

I am deeply shocked!
Have I EVER said anyting stupid??

 

[Clausius2]

I am deeply shocked!
Have I EVER said anyting stupid??

Last times you seem to me like a warrior. Have-you-EVER-read-anywhere-I-said-YOU-have-said-something-stupid?. Tell me where I said that, and If you cannot find it, do not fight against me never again. Unless you are joking, of course.

PD: I'm sure I have said something stupid in this Forum, and the language has to do with it.

 

[arildno]

I might have been joking..(I guess it was a STUPID joke..)

 

[chroot]

College professors are considered authorities, and yet they still make mistakes and say "stupid" things from time to time. Of course, they doesn't mean they should be considered any less an authority. No one should confuse the title of "Science Advisor," or "Professor," with infallibility, as that is an inhuman goal.

- Warren

 

[hemmul]

Look at the blue bar on the top of the first post on the page, just under the page numbers on the far right, it says, "rate this thread."

By the way... actually i can't see it :)
the only things visible are two menus:
1) Thread Tools:
* show printable version
* email this page
* download this thread
* (un)subscribe (from)to this thread

2) Search This Thread
* [edit box]
* advanced options

is it that i have no "rate this thread" option because of limeted profile?

 

[Clausius2]

Confused

I see it. It's a secret vote or somebody is going to know which is my vote?.

Moreover: which are going to be consequences of the rates voted to the author of the thread?

 

[Moonbear]

By the way... actually i can't see it :)
the only things visible are two menus:
1) Thread Tools:
* show printable version
* email this page
* download this thread
* (un)subscribe (from)to this thread

2) Search This Thread
* [edit box]
* advanced options

is it that i have no "rate this thread" option because of limeted profile?

Hmmm...I don't know. "Rate Thread" is just to the right of the two menus you describe in my window. I have no idea why I see it and you don't. Is rating threads a privilege of contributors? I was sure I had seen that bar there all along, even before becoming a contributor or advisor or anything that might have granted me a bonus feature.