Taijitu

City Center => University of the Glorious Revolution => College of Gameplay => Topic started by: Musitant on February 26, 2015, 08:00:15 PM

Title: Taijitu Influence Rankings February
Post by: Musitant on February 26, 2015, 08:00:15 PM
(http://cdn.searchenginejournal.com/wp-content/uploads/2012/05/shutterstock_72009739.jpg)

Influence Rankings

Obviously, it is incredibly difficult to put any number to somebody's influence. Influence is a very difficult thing to capture, as it is more than just how much somebody speaks and is in actuality how much the words affect the listeners. Because measuring listener effects is next to impossible, we're forced into something that doesn't take into account quality so much as quantity.

What I did was, for the month of February, dig in to the Ecclesia and count posts. Depending on the contents of the post, every member got a different amount of "influence points."  I didn't count any votes after successful motions and I ignored World Assembly, Citizen Applications, and the Deployment thread as that was not a discussion of a proposal in February, rather a status update. I also counted public votes in February. The breakdown of points follows.

New Topic10 Points
Successful Motion to Vote3 Points
Seconding Motion to Vote2 Points
Other Discussion1 Point
Vote3 Points
Abstention2 Points

The following rankings are not meant to hurt anyone's feelings or suggest that anyone is better than anyone else. They are merely a fun thought experiment not to be taken too seriously. Without any further ado, I give you Taijitu's February Influence Rankings as of February 26.

Taijitu's February Influence Rankings


6
Cormac
(http://forum.taijitu.org/index.php?action=dlattach;attach=2049;type=avatar)

4
Delfos & Allama
(http://forum.taijitu.org/index.php?action=dlattach;attach=1980;type=avatar) (http://forum.taijitu.org/index.php?action=dlattach;attach=2048;type=avatar)



3
al 'Khem
(http://forum.taijitu.org/index.php?action=dlattach;attach=2086;type=avatar)

2
Gulliver
(http://forum.taijitu.org/index.php?action=dlattach;attach=1543;type=avatar)

1
Myroria
(http://forum.taijitu.org/index.php?action=dlattach;attach=2089;type=avatar)



Full rankings & points:
Myroria62 Points
Gulliver42 Points
al 'Khem26 Points
Delfos22 Points
Allama22 Points
Cormac20 Points
Dyr Nasad13 Points
Bustos12 Points
Musitant12 Points
Pauline Bonaparte11 Points
Funkadelia9 Points
Of The US7 Points
St Oz6 Points
Letonna6 Points
The Church of Satan4 Points
Eluvatar4 Points
Alina3 Points
Stone Shark3 Points
Rapture3 Points
Erno3 Points
Enchancia3 Points
Wast2 Points

Questions
Personally, I'm left with a lot of questions I have difficulty answering. Are these rankings fair? Do I put too many points on voting or discussion? Should I treat all discussion equally or go even further into qualifying which discussion deserves more points? Is this encouraging to the Ecclesia? Is this a good system to compare activity in the Ecclesia month-to-month? Do you guys want to see more?
Title: Re: Taijitu Influence Rankings February
Post by: Myroria on February 26, 2015, 09:00:21 PM
WOOOOO MYRO NUMBER ONE!

In all seriousness this was a very interesting lecture. I will comment more once I'm at home and not on my cell phone.
Title: Re: Taijitu Influence Rankings February
Post by: Khem on February 26, 2015, 09:06:00 PM
I love this but then didn't realize I was so influential :tai:
Very very interesting. Does this make me part of the Cabal?  :keke:
Title: Re: Taijitu Influence Rankings February
Post by: Bustos on February 26, 2015, 09:11:08 PM
 :shrug:  To me, this means who are the most active when it comes to the voting process.  The more active you are, the more points you get.

An interesting take on it though.
Title: Re: Taijitu Influence Rankings February
Post by: Musitant on February 26, 2015, 10:42:25 PM
Points from voting are around 40% of total points, though they're a lesser percentage as we get higher in "influence." I figured that voting is the actual decision making process in the Ecclesia, and so it was considerably influential. The most points anyone received from voting was 9 points, and only five people did so.
Title: Re: Taijitu Influence Rankings February
Post by: Bustos on February 27, 2015, 12:48:09 AM
I said 'voting process' meaning your whole calculation.  Just making a post gets a point, whether or not it 'influenced' the resulting outcome.  More active you are within these threads, the more posts you can make.  Considering almost everything gets voted on, the first to say it and the first to second the motion, gets points, which is helpful if you're actively watching the threads.  Just making a topic is 10 points, whether it resulted in anything or not.

The fact Eluvatar only got 4 points by your calculations as one of the most, if not the, most influential people in Taijitu, demostrates that it's really forum activity of the voting process you're really measuring.

Not trying to put you down.  The fact you made this is cool and well worth the read.  It definitely made me think, and those are my favorite kinds of posts.  I'm just saying I think the results of your calculations doesn't support the conclusions you've made.
Title: Re: Taijitu Influence Rankings February
Post by: Musitant on February 27, 2015, 01:34:21 AM
Do you think Eluvatar was one of the most influential people in the Ecclesia this month? Has he been influencing things outside of the forums? I'm probably going to update this and make it more rigorous in the future (might count discounted WA posts as well) and if you have any advice on future calculations I'd love to hear them.

Honestly, I calculated all attempts at influencing the Ecclesia in February. If two people are trying to influence the Ecclesia and one of them makes posts and the other doesn't, I'm going to say that the one making posts is more successful at being influential. It's impossible to truly measure the effects of one's words so I mostly measure the volume of one's words and assume a correlation. The first step to being influential is showing up.
Title: Re: Taijitu Influence Rankings February
Post by: Musitant on February 27, 2015, 04:24:38 AM
Apparently this is providing the wrong incentive for some people. I thought this might encourage more discourse in the Ecclesia or at least provide an interesting conversation, but it appears this is causing more spam and low effort answers. Rest assured, the next time I do this I will break down "Other Discussion" into "On Topic Discussion," "Agreement Posts," and "Joke Posts."

However, considering that people weren't aware of anyone looking into their activity, the quality of posts in the Ecclesia was relatively strong before today and so I don't think the omission of that distinction had much of an effect on the previous analysis.
Title: Re: Taijitu Influence Rankings February
Post by: Eluvatar on February 27, 2015, 04:48:30 AM
Disappointing, but probably unsurprising.
Title: Re: Taijitu Influence Rankings February
Post by: Gulliver on February 27, 2015, 04:57:17 AM
Very interesting. Have you considered implementing these calculations as code? Doing so would let you play around with the weights and compare the outputs easily, I wonder if a different weighting scheme would produce similar or different results.
Title: Re: Taijitu Influence Rankings February
Post by: Myroria on February 27, 2015, 05:13:23 AM
Musitant, I told you earlier I would post a longer reply to this when I was able. I deeply regret that I must devote some of this time not to critiquing your work, but to scolding a petulant child.

I suppose I'll start with the scolding first. I won't pretend I'm not angry by what happened today, and Bustos, your actions today are what angered me. When this University was created it was created to be a place where people could post their knowledge without fear of being mocked for it. You disagreed with the content of Musitant's post. That much is clear. You posted your issues with it. I, and I think most Taijituans, have no problems with people posting issues they may have with another person's lecture. This is encouraged.

But to spam the Ecclesia as if you're "keeping score" is a petulant way to act. It shows a disregard for not only Musitant's message, but his intention as well. His intention was sharing knowledge, and to discourage this in any way is to discourage people from using this University as it was intended. To be frank, it also shows how little you value the direct democracy itself - that you would so lightly use it to prove a point.

Disagree with Musitant's message all you want. But do it the way adults act - by debate, and not the way children act - by silly defiance.



I would like to speak now about the topic of this thread.

I would first like to thank Musitant for his time in putting this together. You are right to say it is hard to define influence in a meaningful way - I think that perhaps, to sooth sore egos, an alternative such as "civic activity" may have been more appropriate. But that is neither here nor there. No matter how you name the metric, it is still the same thing.

I think that assigning numeral values to each action taken in the Ecclesia is the most sensible way to go about this. Though, as you said, some of that je ne sais quoi of influence is bound to be lost by converting this to mere numbers, it seems the most practical way to do it.

When looking at the value assigned each action, the main critique I would have is that "starting a thread" may be weighted too heavily. I wouldn't be surprised to learn if Gulliver and I scored so highly only because we started the most threads. In most cases, ideas are briefly discussed in our IRC channel, #taijitu on esper.net, before being put on the forums. These brainstorms are typically a very communal affair, and often Gulliver or myself are the ones who happen to post it in the Ecclesia. It can feel a little unfair to see us scored so highly when really, other than posting topics, I'm far less active in discussion than Delfos (in my opinion, at least).

Despite this, though, I see why one may rank starting a thread so highly. I would probably suggest it remain the highest-scoring option, but to weigh it at perhaps 5, or even 4, instead.

To give my opinion to some of your questions at the end:

I think it would be hard to define which types of discussion get more points, but I would like to see some ideas.

I think a monthly report like this would definitely encourage the less-active among us in the Ecclesia to post more.

I think it serves as a good metric of Ecclesiastical activity, and who is doing most of it, but I would be cautious with making any solid statements about how influential certain Taijituans are with it.

I would love to see more!

Title: Re: Taijitu Influence Rankings February
Post by: Gulliver on February 27, 2015, 05:26:18 AM
Reading Myroria's followup, a thought occurred to me. "Influence" is in some degree fundamentally about impressions people make on other people. Were we conducting this study on, say, the US House of Representatives we could in conjunction with it run a poll on which members people consider the most influential, and use it to gauge the success of the model. We could attempt something similar here, but with such a small sample size the poll data would likely be very noisy and maybe downright unhelpful. It may, thus, be impossible or very difficult in such small setting to judge formally how accurate any model is.
Title: Re: Taijitu Influence Rankings February
Post by: Bustos on February 27, 2015, 07:07:43 AM
I do love how I am singled out for this as if I am the only one.  Not surprised.
Title: Re: Taijitu Influence Rankings February
Post by: Funkadelia on February 27, 2015, 07:18:44 AM
I really like this idea. I've never seen it actually done before anywhere, so it's a neat process you used to measure the influence of people in Taijitu based on their pure activity. I can't wait to see more stuff from you.
Title: Re: Taijitu Influence Rankings February
Post by: Delfos on February 27, 2015, 12:14:44 PM
All in all this is the power and flaw of Statistics. They do not depict a real portrait, they never do, they are a tool to try to understand and try to balance what we may think is unbalanced. For instance: none that I have proposed since Ecclesia started was passed (as I proposed at least), yet I feel I have deeply contributed for many of the ideas in place. Whether people recognize or not that most of us have influenced every idea since Ecclesia (and that is what makes us strong), even some of the "trolls" have a design.

Maybe a way to compliment this statistic is an inquiry to see how people feel about the contributions and that would add points up or down and have a less "mathematical" and more "perception" based counting.
Title: Re: Taijitu Influence Rankings February
Post by: Myroria on February 27, 2015, 01:59:26 PM
I think Delfos makes an excellent point. While I worry about adding too many things to score, an idea would be a "Proposal that gets added to final bill" metric. I know that, personally, I've added tons of people's suggestions to billls I've drafted and I would hate to get all the credit for that.
Title: Re: Taijitu Influence Rankings February
Post by: Eluvatar on February 27, 2015, 04:41:14 PM
I think Delfos makes an excellent point. While I worry about adding too many things to score, an idea would be a "Proposal that gets added to final bill" metric. I know that, personally, I've added tons of people's suggestions to billls I've drafted and I would hate to get all the credit for that.

That really could make this even more interesting. :)
Title: Re: Taijitu Influence Rankings February
Post by: Musitant on February 27, 2015, 04:42:14 PM
Gulliver, you're probably right about the code. I kept some raw numbers so that I could lightly experiment with what you were talking about with differing weights, but not all of them. Honestly, I stayed away from code because I'm such a bad coder I think I would've spent more time making the code than I would've saved by using it. Another problem is that I also only counted "successful" motions and seconds, so it would be difficult (especially for my dumb brain) to get a program that recognizes the success of a motion to vote as people sometimes continue threads afterwards. I also think a poll isn't a bad idea for gauging how well this did. I think to get at some of the noise we could aggregate rankings by those who were ranked. I might make one in the following days.

Myroria, looking back you're probably right about the thread starting. That may need to be downplayed in any further rankings, though I will still keep it as the most influential thing one can do. The reason I tried to stay away from too much of whether or not a proposal was successful was my own impatience. There were too many proposals up in the air that I thought were incredibly likely to pass and so I didn't want to wait until March in order to allocate points.

Bustos, you're right it wasn't just you. You just made it really obvious with your "+1" stuff. Also, just to let you know, a lot of the posts you included a "+1" would never have counted even if I didn't distinguish between regular and joke discussion. You posted a lot in things that were not proposal discussions and you posted in proposal discussions that already had finished. The first successful "second" is the last post that received any points in any thread.

Funkadelia, thanks! I'll keep doing it.

Delfos, I completely agree. I just hope that this gets at some of what I was trying to capture, which I think it does. I will definitely have a poll and see how those rankings compare to this. Though I feel like polls are subject to even more issues and problems than the ranking system, if they match up somewhat it's a vote of confidence for both.
Title: Re: Taijitu Influence Rankings February
Post by: Khem on February 27, 2015, 06:47:29 PM
Perhaps a poll of all current Ecclesia members with condorcet style ranking to gauge our perceived influence?
Title: Re: Taijitu Influence Rankings February
Post by: Bustos on February 27, 2015, 08:31:04 PM
Perhaps a poll of all current Ecclesia members with condorcet style ranking to gauge our perceived influence?

Now you're talking.
Title: Re: Taijitu Influence Rankings February
Post by: Delfos on February 27, 2015, 08:46:30 PM
I don't know how that differs from an election poll in terms of subjective preference, if you want more objective results you have to ask objective things. Either who do you think contributed for said issue, who has contributed for activity, who do you think is most active, things like that, this is a poll about perception on influence, not who you think is or should be the most influential. They might not differ much but you should be objective, there are a lot of techniques in regard to inquiries and there's plenty of free websites for said inquiries:

example that I've used before:
https://www.wufoo.com/pricing/
Title: Re: Taijitu Influence Rankings February
Post by: Khem on February 27, 2015, 09:04:14 PM
I don't believe there is any manner to objectively note influence. One cannot see the ways in which words of one take hold in another's mind. However we could create subjective methods ad infinitum with no bearing on absolute truth they could be amusing none the less.
Title: Re: Taijitu Influence Rankings February
Post by: Dyr Nasad on March 01, 2015, 03:49:17 PM
This is certainly interesting. Most previous attempts at quantifying influence in NS have been through surveys, and having some objective data presents an alternative way of looking at what is otherwise an incredibly nebulous topic. As others have stated, the degree to which posts and visible discussion influence the course of legislation is often not clear. However, this ranking is a good place to start; you have generated a significant response from everyone here, and adding weightings and more specific categories to your ranking would create an even better product.

People always complain about the shortcomings of a pure quantitative or a pure qualitative approach to these analyses - so do both! A future ranking would be even more interesting if it included some measure of perceived influence that is missed by a strictly objective method.
Title: Re: Taijitu Influence Rankings February
Post by: Musitant on March 02, 2015, 04:24:02 AM
I took multiple people's (Gulliver, al 'Khem, Bustos) advice and made THIS SURVEY (http://goo.gl/forms/y0Vsci7hG4). I agree with Delfos that it will entirely be people subjective opinions and that personal biases and preferences will be the real result of this survey. However, it will be interesting to see where this survey will differ from a quantitative approach and so I'm interesting in seeing the results. Keep in mind that I won't view your forum name at the same time as your responses and that I'll never publish your responses attached to your name.
Title: Re: Taijitu Influence Rankings February
Post by: Delfos on March 02, 2015, 09:29:16 AM
I took multiple people's (Gulliver, al 'Khem, Bustos) advice and made THIS SURVEY (http://goo.gl/forms/y0Vsci7hG4). I agree with Delfos that it will entirely be people subjective opinions and that personal biases and preferences will be the real result of this survey. However, it will be interesting to see where this survey will differ from a quantitative approach and so I'm interesting in seeing the results. Keep in mind that I won't view your forum name at the same time as your responses and that I'll never publish your responses attached to your name.

* me opens link, finds out I gotta digit my nation nation and gets sad like a kitten without water to drink *

* continues, it's all based on names, gets more sad D: *

I suggest you make the same list with scales 1 to 3 or 1 to 5, it's more fair than ranking ordinarily each one, I got to the 2nd page and either forgot the names or didn't care anymore.
Not at all influenced | Was Present | Participated | Insightful opinion | Paramount ?