Should people only score edges, and not nodes?

[keyserj]

TODO: create an Ameliorate Topic reflecting the tradeoffs described here and in the thread around this comment #478 (comment)

Describe your issue

In the cars-going-too-fast example, you might score the Detriment "pedestrians might get hit" a 9 to convey that it's a big concern, but the edge where it's "created by" Problem "cars going too fast in my neighborhood" could be scored a 3 to convey that, while the Detriment is a concern, you don't think it's actually created by the Problem.

I think being able to make this distinction is important, because I feel that disagreements often miscommunicate such distinctions (e.g. imagine one party accusing another of not caring about people getting hit, when it's specifically that they don't think that cars actually cause the problem).

Unfortunately, I also generally have been scoring nodes within the context of the Topic. So in the above case, since the Problem is the Topic, if I didn't think the Detriment was relevant to the Topic, I would score it low. I feel that this is also a useful thing to have, because otherwise people can always come up with some reason why the Detriment is important that's irrelevant to the Topic, which isn't useful for the discussion. But this conflicts with the above intention.

I think this conflict creates ambiguity and is therefore pretty important to deal with.

Solution you'd like

No response

Alternatives you've considered

It also seems possible that the scores don't need to be unambiguous, that the score with ambiguity is sufficient enough to prompt discussion/addition of justification.

Additional context

No response

Technical ideas and questions

No response

[keyserj]

Idea: nodes should always be scored based on importance within the context of the topic, and never exclusively to the connected parent node(s) (that's what the edge score is for).

[prototyperspective]

1. It would be still ambiguous as one can differently interpret the context of the topic or how importance is assessed within it (e.g. importance to solving the problem vs importance in terms of problem severity) 2. I don't think people would understand (e.g. not read some info how the rating is supposed to be used and rate nodes connected to just one node for its importance in relation to that node like the edge is supposed to be rated) and/or rate it accordingly but apply many different notions, misinterpret things and be confused.

I wouldn't know how one could make an additional relevance/importance within the broader problem context rating a well-working useful thing – maybe there is but imo it seems more likely it will for most users be more clutter and make things harder to understand. Maybe one could have these extra ratings only on problem nodes so assessment of the importance of [solving] that subproblem in context of the overall larger problem.

[keyserj]

It would be still ambiguous as one can differently interpret the context of the topic or how importance is assessed within it

I think this ambiguity is reduced by considering the type of the node being scored. Importance of a problem's Detriment would indicate severity, importance of a solution's Benefit would indicate solving.

There would still be ambiguity remaining, but at least there's an outlet for intuition that I think is not covered sufficiently by just edge scores e.g. when a relation seems missing, but you can't quite identify it yet (I think we also have an open discussion in #478 about the worthwhile-ness of this example but I'll respond to that soon in that discussion). And a difference in ambiguous scores can still help prompt further reasoning to clarify that ambiguity (and, with the ambiguity clarified, the original difference in intuition can then be clarified, if there is one).

[keyserj]

@prototyperspective here is some continued thoughts in reply to #478 (comment). I think this stuff is worth discussing at some point but I agree that it may be better put off until there's more usage of the tool to have feedback for the scoring.

---

I meant that people should specify why they disagree instead of just rating things with gut-feelings or hivemind-voting

I see, thanks for clarifying - I was assuming you meant specifying reasoning before disagreement is identified. In this case, it seems like we both would support the idea of prompting for justification when you score differently from other people's scores.

I don't see how also rating nodes instead of just nodes in relation to other nodes (edges) helps with [reducing time spent misunderstanding disagreement]

Rating nodes can convey suspected importance beyond the relations in the diagram, which is one thing that people can disagree about. And since these intuitions can be abstract, this seems more likely to be a thing that people don't think to bring up, leaving a disagreement unidentified.

I think the simple way I justify this to myself is by asking "Can I reflect not only all of my knowledge, but also all of my intuitions in this map?" If only knowledge is reflected, then the map is not a wholistic representation of all relevant information, because intuitions are a part of understanding and are used for decisions.

Ideally I agree that intuitions would be justified and concrete, but realistically, justifying and making-concrete of intuitions can be time-expensive, and shouldn't need to be done before knowing that people disagree, and that can't be done unless there's a way to convey the intuitions before having justification.

I agree that potential ambiguity can make this more detrimental than beneficial, and that it requires more thought.

even all contributors to a Wikipedia article already know of / agree about something that often should still be in the article since the article is intended for a general audience

This is a good point, but

1. I'm specifically talking about relying on scores when justification isn't known yet. If you already have reasoning, I think yes you should proactively identify it (especially if it's for a wiki-style map that could be encountered by people that don't have the same context as you).

2. If viewers want to briefly understand a map, they should be able to quickly read through it. But if they want to thoroughly understand it (hopefully at least policy makers would), they should be scoring it too, identifying their own disagreement, and providing opportunity for being explicit about the disagreed-upon things.

It does the opposite of that because people may just rate a node instead of adding a critical node like a Detriment

I assume you're saying that this would happen because the score would act as an outlet to relieve people of desire to communicate reasoning. Maybe this highlights the importance of automated prompting for justification based on disagreement. Or even something like "request justification" (for if someone notices someone else disagreed and didn't provide reasoning).

there may already be a critical node attached to it so people mistakenly think that's why the node is rated so low by some

If that other node (and/or its edge to this node) was scored highly, that difference in score should clarify that there's some missing information.

Instead of just downvoting every relevant node without even specifying why

This sounds more like bad faith usage of the tool, which is lower priority right now, but I do think that only scoring edges would still have this problem, and that there are other ways to potentially counteract it. I think establishing a credibility system seems like it would help. Perhaps e.g. unanswered "request justification"s could detrimentally impact someone's credibility.

Do you think people will carefully check each and every contributor's ratings and then ask them why?

No, but I think tooling could be added to indicate and prompt-to-address these inconsistencies.