Requirement for beta kata approval needs to be more transparent

[Voileexperiments]

I constantly get asked by people I know "when will my beta kata get approved?", and honestly, not even I know either.

I think a comprehensive list of "score required to make a kata able to be approved" along the table that shows all honor sources and privileges would be helpful to pretty much everyone, especially power users. There are several thing we'd really want to know:

1. How many score (upvotes) are required before I can approve a kata?
2. How does 'somewhat satisfied' and 'not satisfied' affect this process? Because I've seen katas like this which has way too many solves and upvotes and nevertheless is still showing "Testing & feedback needed".
3. How does translations affect kata approval? Because again, while I appreciate GiacomoSorbi 's effort to translate lots of beta katas in many languages, approving them seem to just make a kata impossible to move past beta, which makes some of us joke that 'don't approve translations before your kata is approved'.
4. How much extent does the number of already published number of katas affect kata approval? I've seen a table about it somewhere on CW or here (possibly by @kazk), but I couldn't find it again anymore. Having this information handy would be really helpful.

┆Issue is synchronized with this Clickup by Unito

agreed, i hated the process because it's not clear at all, i think it should be automated

[kazk]

@Voileexperiments

I've seen a table about it somewhere on CW or here (possibly by @kazk), but I couldn't find it again anymore. Having this information handy would be really helpful.

The following table is from #866 (comment).
Note that I never got a reply from @jhoffner if this is correct interpretation, so this might be wrong.

Level	Satisfaction %	Required Votes by # of Approved Kata
		<10	<20	<30	30+
White	90+	12	10	8	6
Yellow	85+	10	8	7	5
Blue	75+	8	6	5	4
Purple	70+	5	4	3	3

Based on #757 (comment) and rounding to nearest integer.

---

The starting point of the new documentation is awaiting for @jhoffner's review and licensing. We can add more information about the beta process there.

@kazk this should be implemented as an algorithm

[jhoffner]

How many score (upvotes) are required before I can approve a kata?

This value is dynamic based off of overall ranking feedback and what voting privileges someone has. You can have up to 4x voting power. @kazk's chart looks to be correct.

How does 'somewhat satisfied' and 'not satisfied' affect this process? Because I've seen katas like this which has way too many solves and upvotes and nevertheless is still showing "Testing & feedback needed".

It only affects the satisfaction score. If a score is too low, a kata cannot be approved. The limits here are also based off of ranking feedback.

How does translations affect kata approval? Because again, while I appreciate @GiacomoSorbi 's effort to translate lots of beta katas in many languages, approving them seem to just make a kata impossible to move past beta, which makes some of us joke that 'don't approve translations before your kata is approved'.

If anything translations should help the process, because more languages means more users solving which means more feedback/voting. How have you seen translations hurting the process? You will note g964's kata tend to come out of beta very fast and it's because he translates his kata into so many languages.

How much extent does the number of already published number of katas affect kata approval? I've seen a table about it somewhere on CW or here (possibly by @kazk), but I couldn't find it again anymore. Having this information handy would be really helpful.

@kazk's chart above shows. The more kata you have approved, the less restrictive the system is on approvals since you have shown (in theory) that you consistently produce quality content.

[Voileexperiments]

@kazk @jhoffner Thanks for the information. However, based on my actual observation the above system is not working quite well in practice:

• For high-difficulty katas the people solving it are power users anyway, so you can expect them to behave as ideal person and give straight 100% score before approval
• For the lower end of katas (mostly white katas), yes, there are lots of solves, but also people that put in "somewhat satisfied" votes (I guess they think the kata is too easy) and then run away forever. These dead weight easily drags down the overall score, and not even 4x votes can counteract that. In practice, white katas usually stay around 80% score even for good ones, so lots of them are pretty much doomed unless all the votes are cleared.

So, I think maybe the required percentage should be reversed (75/80/85/90 instead of 90/85/80/75)? Because based on actual statistics, clearly white/yellow katas have lower overall score percentage than blue/purple katas.

[kazk]

@jhoffner

How have you seen translations hurting the process?

When the translations are low-quality and the translator isn't responsive. No notification, too busy, no longer active, etc. This is obvious, but the damage is catastrophic when one translator produces multiple low-quality translations and doesn't maintain them.
g964's kata doesn't suffer from this because the author is the translator for most of the languages and maintains well.

---

@Voileexperiments

In practice, white katas usually stay around 80% score even for good ones

The satisfaction % tends to drop after the approval regardless of the color and usually the easier ones drop more, but this is irrelevant when discussing about approval process.

Do you think a 8 Kyu kata with 75% satisfaction rating should be approved?
Note that this number is during beta and votes are mostly from "power users".

I agree that the current system isn't ideal, but I haven't seen a proposal that's realistic to implement and works better.

[Voileexperiments]

Do you think a 8 Kyu kata with 75% satisfaction rating should be approved?

Ideally the one who approves the kata (i.e power user) should check the obvious things about the kata before approving. But it seems that some people approve beta katas without even first solving said kata :V The approval process is not as light as many of us thinks, and when approving a kata we really should be cautious and careful.

it's easy to fix -> have a threshold satisfaction and threshold number of upvotes -> if both thresholds are met approve -> if a kata or a translation fall beneath put into beta (if translation just put the translation in beta, if that's not possible then delete the translation)

[10XL]

Shouldn't the percentage that votes affect satisfaction rating scale like points? Meaning someone with 4x vote power would affect the satisfaction rating 4x.

For example, a new user's(1x vote power) negative vote affects the satisfaction rating by 25% of a user with 4x vote power. So if a kata that has a 100% satisfaction rating with 9 votes gets a negative vote from me, it will have 10 votes and a 90% satisfaction rating. If instead the kata received a negative vote from a new user, it would have 10 votes and a 97.5% satisfaction rating since he would have 4x less vote power than I have.

[DonaldKellett]

With regards to beta Kata approval, I would like to bring this suggestion back into the light. The system as described in the proposal should minimize the chances of an irresponsible power user / moderator approving a substandard Kata that is overrated by "new" users.

[DonaldKellett]

it's easy to fix -> have a threshold satisfaction and threshold number of upvotes -> if both thresholds are met approve -> if a kata or a translation fall beneath put into beta (if translation just put the translation in beta, if that's not possible then delete the translation)

Automating the process seems like a viable idea. @jhoffner perhaps this should be considered in the future. Only thing to note though if the process really does become automated - the thresholds for "automatic approval" would need to be even higher, e.g. 95%/90%/85%/80% and the minimum number of upvotes would have to at least double. We don't want the system to "objectively" approve a substandard Kata that just happened to receive an influx of upvotes from inexperienced Codewars users subjectively voting on the Kata based solely on their enjoyment/satisfaction of solving it and not on the quality.

@ericswenson15 As for putting a Kata back into Beta when the satisfaction score drops below the threshold, consider the impacts it would have on the users' ranks and honor points. If a high-kyu approved Kata is put back into Beta by the system then the users who have solved it would immediately lose the huge amounts of rank and honor they gained when solving it as an approved Kata which would lead to a large number of frustrated and confused users.

true it would impact their rank and honor negatively, they would drop in honor for sure. very necessary if they write a kata that doesnt meet the standards of other users on codewars

this group would not be large, and is currently greatly outnumbered by those experiencing frustration with the approval process. also it's totally necessary, if they write a kata which used to be good enough and now no longer is, then their honor should be docked, and restored if they get it back out of beta. it's idea capitalism, survival of the fittest