Template talk:Ratings

Perfect vs Excellent

From Talk:Stereoscopic 3D Support and Compatibility/Sandbox, it occurred to me that we should bring one of the things proposed there over here! We should change "Perfect" to "Excellent". Excellent provides a way to allow minor errors in, such as shader compilation stuttering that is minor for every single game, without it interfering with the rating. And it should finally ease the "this is not perfect!" arguments, since the bar would be lower! It's also a drop in change for the 5th star, and very simple for us to do. What do you think guys? - MaJoR (talk) 10:13, 29 November 2015 (CET)

One thing to note is that the emulator would need to be updated too. I wanted to ask here first though. I'm pretty sure it will go well there, but it's something to consider when thinking about this. - MaJoR (talk) 11:29, 29 November 2015 (CET)

I very much dislike revising things so that the top rating isn't perfect. If some title is not perfect, then there should still be things to be worked on in the emulator related to it, and I don't what that to be lost as we start applying 5 star to imperfectly emulated titles. In theory we should have already resolved "shader compilation stuttering" resulting in imperfect ratings by migrating those problems to "Emulation Info" rather than under "Problems". Kolano (talk) 17:44, 29 November 2015 (CET)

We should be careful with "Perfect" rating, like, should we consider a game that need specific settings as "Perfect" or if a game needs settings that'll make it run very slow but accurate, should it get a Perfect rating too? That's the main issue I'm seeing with our current system, we lack a well defined set of aspects to quickly define what specific rating a game should get. Our current definitions are too vague and leads to different understandings, like this particular case... - Jhonn (talk) 18:23, 29 November 2015 (CET)

That is kind of what this is meant to deal with. We've had games that were perfect, and when a super minor bug is discovered, even though it was there all along, it is demoted to 4 stars. By lowering the standard just a little bit, a lot of problems will be reduced. "Perfect" is an impossible standard! A little give will smooth out a lot of issues in the rating system, imo. - MaJoR (talk) 08:43, 30 November 2015 (CET)

"Perfect" in technical sense, yes, it's impossible to emulate a game like perfect. When it comes to rate the compatibility with a game, it's true that when there's a minor (yet legit) bug discovered then it will receive 4 stars in an instant, it's also far easier to comprehend and manage. Just wait until that game has zero active problems in the "Problems" section. Be grateful that there is Emu Info section exists for non-genuine problems! It's matter of what in sense does the term "Perfect" refer to. And for some reason the term "Excellent" doesn't get along well with "Playable" and other terms in the compatibility rating list. Lucario (talk) 11:02, 30 November 2015 (CET)

Why would Excellent not get along with Playable? :/ It seems fine to me. - MaJoR (talk) 14:27, 30 November 2015 (CET)

The definition behind five rating sets feels distinctive from each others. I think it's perfect (not technical of course!). The definition of "Excellent" will blur the bar between "Playable" and "Perfect" and will not end well as if there's debate between whether the problem is quite "minor" or not. We shouldn't be splitting hairs there. With "Perfect", anytime there is a problem then let the other four rating sets do the job. They're based on where the point it crashed during the emulation. If no crash, gameplay progress fine, but still has problem, "Playable" it is. Lucario (talk) 07:34, 2 December 2015 (CET)

Hmm... How about this - we still have minor issues demoting from 5 stars to 4, so perfect would more or less just be renamed. But even like that, I still think excellent is a good thing to move to. Excellent would more or less be the same as Perfect, but with a small change - the global bugs, undiscovered bugs, and all of those things we ignore and allow us to call it perfect just because the problems area is empty? It's tolerable now, because it's NOT PERFECT! Perfect is perfect, and as long as we have shader compilation stuttering, *nothing can be perfect*. Moving to this would give us the wiggle room to actually say something is excellent and allow us to actually mean it. It wouldn't be about changing our rating system at all, it would only be about changing the word we use to describe the 5th star to better reflect how we are using it. - MaJoR (talk) 17:30, 2 December 2015 (CET)

I wonder if the shader compilation stuttering is pc performance dependent? Hypothetically, there is one badass PC with latest gen specs, will we still see stutter there? If not, then no way it will hurt the compatibility rating of the game. We shouldn't mix pc performance with compatibility rating system. Anyway, there are side effects to consider about when changing from "perfect" and its description to "excellent". People don't have the exact same mindsets as what we're having here (I'm okay with "Excellent" as long as it's exactly the same as "perfect" in the compatibility rating system, just becoming less too technical behind its meaning). Some may rate the game as 5 stars despite there's some active issues in the problem section that may seem extreme minor to them but didn't know that the 5 stars was only when the problem section is empty. Lucario (talk) 09:49, 3 December 2015 (CET)

A computer with the absolute latest and best parts overclocked to the maximum will still see shader compilation stuttering. It's a fundamental difference between how the GC/Wii and PCs work, so it isn't related to performance at all. Another example of this is issues with forced perspective to get around CRT timing issues, which creates subtle bugs in many GameCube titles, such as one pixel of masking being wrong, one pixel high jittering (which is a bigger problem worthy of the problems area), and other minor things without easy answers. All of these minor problems I'm fine saying a game is "Excellent", but "Perfect"? It feels wrong to me to ignore those things... - MaJoR (talk) 10:10, 3 December 2015 (CET)

What about that PC from the year of 3,000? Just kidding! I read somewhere that shader caching will reset every time there a new GPU driver installed. It's still PC dependent in some sense, no? I figured out why using "Excellent" in compatibility rating system feels funny to me. Dolphin is never perfect, that's certainly a given. When it comes to "Compatibility rating"? It just asks if a game works fine on Dolphin, so if there's a crash, glitch, or such thing that disrupts player's gameplay progress, rate it. "Excellent" is probably far more appropriate term when it comes to "Accuracy rating" or so what we'd call it. I actually don't consider the accuracy flaw a "problem worthy" if it doesn't disrupt gameplay progress at all (or catching their attention in x1 native IR). Unless I'm not understanding you correctly, does the one pixel high jittering still happening in x1 native IR? Any subtle bugs only visible with IR higher than x1 are not the real emulation problem. People can still provide patch for it in the Enhancements section. Tales of Symphonia#Reduce Double Image is one good example. Lucario (talk) 01:07, 5 December 2015 (CET)

"When it comes to "Compatibility rating"? It just asks if a game works fine on Dolphin, so if there's a crash, glitch, or such thing that disrupts player's gameplay progress, rate it." By that definition, once a game is completable, it is "perfect". That's ridiculous, and not what anyone expects by the definition of perfect. The plain common-sense interpretation of perfect compatibility is no problems at all ever under any circumstances period. That you can play this game and never encounter a bug whatsoever. I absolutely do not think that the name "compatibility rating" vs. "accuracy rating" influences the definition of the word "Perfect". Not to mention we are using it as an accuracy rating and a compatibility rating in one measure. Anyway, I'm not going to talk about the details of these bugs or try to justify these bugs to you, but if you actually thought Dolphin was emulating perfectly, you need to move past that. It does not, and never has, and any dev would tell you that. There are fundamental problems with how Dolphin emulates the GC/Wii on PCs that cannot just be worked around. There is absolutely no game that qualifies as perfect.

Anyway, this conversation is starting to go into circles, so I'm going to push it toward a conclusion and ask the other admins to weigh in. My point is simple: Dolphin can never be perfect! And yet we are ignoring global problems and unknown bugs to give games the immutable "perfect" rating. By changing from Perfect to Excellent, and do no other change to our ratings template, we are making the wording used in the compatibility rating better match how we are using it! And that's it! We'll actually be able to tell someone that a game is excellent without lying to them. That is what bothers me about "perfect" and why I want to make the change. - MaJoR (talk) 09:48, 5 December 2015 (CET)

One other area to remember in this renaming is the 5 star {{VersionCompatibilityVersion}} templates. Kolano (talk) 23:20, 2 December 2015 (CET)