Template talk:Testing: Difference between revisions

From Dolphin Emulator Wiki
Jump to navigation Jump to search
(Username rename (MaJoR => MayImilae))
(19 intermediate revisions by 2 users not shown)
Line 5: Line 5:


== Android Testing Results ==
== Android Testing Results ==
I recently saw some users posting test results of Dolphin for Android in the forums. Should we start including them in the wiki? [[User:Jhonn|Jhonn]] ([[User talk:Jhonn|talk]])
I recently saw some users posting test results of Dolphin for Android in the forums. Should we start including them in the wiki? [[User:mbc07|mbc07]] ([[User talk:mbc07|talk]])


We can, I may need to make some template edits to handle categories, documentation updates, and etc for such. [[User:Kolano|Kolano]] ([[User talk:Kolano|talk]]) 08:28, 1 May 2014 (CEST)
We can, I may need to make some template edits to handle categories, documentation updates, and etc for such. [[User:Kolano|Kolano]] ([[User talk:Kolano|talk]]) 08:28, 1 May 2014 (CEST)


The main problem I see is that... will users even know what's in their phones? Sure they will possibly know what their phone is, so I guess that gives us something to work from, but are you ok with getting CPU and GPU information from a phone name? And how to phone names fit in the template? - [[User:MaJoR|MaJoR]] ([[User talk:MaJoR|talk]]) 16:30, 5 May 2014 (CEST)
The main problem I see is that... will users even know what's in their phones? Sure they will possibly know what their phone is, so I guess that gives us something to work from, but are you ok with getting CPU and GPU information from a phone name? And how to phone names fit in the template? - [[User:MayImilae|MayImilae]] ([[User talk:MayImilae|talk]]) 16:30, 5 May 2014 (CEST)


Well, if the user want to upload an Android test then we should require they at least know what CPU and GPU their phone have. Another possible (but a little ugly) solution would be matching CPU and GPU from a table (maybe in a sub template) with phone names to do the thing, so, in the test entry the user would just put something like this: <pre>{{testing/entry|revision=4.0-xxx|OS=Android 4.3|phone=LG G2|result=Black screen, buggy drivers|tester=AndroidUser}}</pre>  
Well, if the user want to upload an Android test then we should require they at least know what CPU and GPU their phone have. Another possible (but a little ugly) solution would be matching CPU and GPU from a table (maybe in a sub template) with phone names to do the thing, so, in the test entry the user would just put something like this: <pre>{{testing/entry|revision=4.0-xxx|OS=Android 4.3|phone=LG G2|result=Black screen, buggy drivers|tester=AndroidUser}}</pre>  
and in the page it would be replaced by the common template (Revision=4.0-xxx, OS=Android 4.3, CPU=Krait 400 Quad Core @ 2.26GHz, GPU=Adreno 330). The main issue with this approach would be the big nightmares we would get, like maintaining the list updated with all of Android phones available, not to mention having a lot of entries for the same phone in the sub template table (Samsung Galaxies for example, flagship models get at least a Qualcomm variant and an Exynos variant in most cases) [[User:Jhonn|Jhonn]] ([[User talk:Jhonn|talk]])
and in the page it would be replaced by the common template (Revision=4.0-xxx, OS=Android 4.3, CPU=Krait 400 Quad Core @ 2.26GHz, GPU=Adreno 330). The main issue with this approach would be the big nightmares we would get, like maintaining the list updated with all of Android phones available, not to mention having a lot of entries for the same phone in the sub template table (Samsung Galaxies for example, flagship models get at least a Qualcomm variant and an Exynos variant in most cases) [[User:mbc07|mbc07]] ([[User talk:mbc07|talk]])


----
----
Line 19: Line 19:


It looks like you did capture some of the mobile CPU/GPU stuff. This also seems fine, though I also have a bad feeling many folks will have no idea what CPU/GPU are in their phones and we'll be cleaning stuff up a lot. I guess that would be dependent on actually getting some Android test results though. [[User:Kolano|Kolano]] ([[User talk:Kolano|talk]]) 15:24, 17 November 2015 (CET)
It looks like you did capture some of the mobile CPU/GPU stuff. This also seems fine, though I also have a bad feeling many folks will have no idea what CPU/GPU are in their phones and we'll be cleaning stuff up a lot. I guess that would be dependent on actually getting some Android test results though. [[User:Kolano|Kolano]] ([[User talk:Kolano|talk]]) 15:24, 17 November 2015 (CET)
: I won't work on this for a while, just realized some other issues we'll certainly have in current design (e.g. two different devices may use exactly the same CPU/GPU and Android version and yet may perform completely different because of OEM shipping older/newer GPU drivers or one including optimizations not available in the other). Including device model would fix that but it would also probably require some redesigns. In other words, at least currently it's not worth the work needed, we still don't have any Android testing entry (on the wiki)... - [[User:Jhonn|Jhonn]] ([[User talk:Jhonn|talk]])
: I won't work on this for a while, just realized some other issues we'll certainly have in current design (e.g. two different devices may use exactly the same CPU/GPU and Android version and yet may perform completely different because of OEM shipping older/newer GPU drivers or one including optimizations not available in the other). Including device model would fix that but it would also probably require some redesigns. In other words, at least currently it's not worth the work needed, we still don't have any Android testing entry (on the wiki)... - [[User:mbc07|mbc07]] ([[User talk:mbc07|talk]])


== Using Variables ==
::We now have one, from a video of 4 Swords on Android, but that's from ~2.5 years ago. [[User:Kolano|Kolano]] ([[User talk:Kolano|talk]]) 13:41, 2 December 2015 (CET)
So I've been contemplating how variables might be used to communicate among templates and get this template to do some more work for us. A couple things to try out...
:::I think I found a way to account for Android test entries, what about adding a new <code>device=</code> parameter to {{tl|Testing/entry}} that get the device model (in the common format <brand> <series> <model> and if necessary <variant> -- but only if that specific variant ships with a different SoC)? When specifying a device this entry would span across CPU and GPU columns (to prevent breaking current table), solving the issue of different phones with the same SoC performing different while maintaining simple for the user (which will need to know only his phone model). Also, a tech-savvy reader could quickly search for the phone model if he wants to know that CPU/GPU that specific device has. I didn't code a sandbox but that's what I mean:
*Setting a "Last Test Release" variable, and using it to flag when test results are out of release order (things like the 5.0-rc releases may muck with such a bit though, since there's a small overlapping range of point in time ids there).
:::{|class="testresults wikitable sortable mw-collapsible mw-collapsed" style="font-size:90%"
*Use the "Last Test Release" variable in [[Template:Testing/end]] to generate categories indicating an age since the last test was performed (i.e. "Tested within 100 revisions", "Tested within 1000 revisions", etc). [[User:Kolano|Kolano]] ([[User talk:Kolano|talk]]) 22:17, 20 November 2015 (CET)
! colspan="6" | Test Entries
|-
! Revision !! OS Version !!  CPU !! GPU !! Result !! Tester
|-
|{{revision|4.0-3423}}
|Windows 10
|Intel Core i7-3630QM @ 2.4GHz
|NVIDIA GeForce GT 635M
|Great
|mbc07
|-
|{{revision|4.0-4636}}
|Android 4.4.2
| colspan="2" | Samsung Galaxy S4 (I9506)
|Good
|SammyDroid
|-
|{{revision|4.0-7423}}
|Windows 8.1
|Intel Core i5-5200U @ 2.2GHz
|NVIDIA GeForce GT 920M
|Great
|BroadwellPC
|-
|{{revision|4.0-8031}}
|Android 5.0.2
| colspan="2" | Sony Xperia Z3 Plus
|Good
|Xperia007
|-
|{{revision|4.0-8031}}
|Android 5.1
| colspan="2" | Motorola Moto X Play
|Good
|MotoDroid
|}
:::It would also simplify category generation (we would need to handle only device brand instead of mobile CPU/GPU). So, any thoughts? - [[User:mbc07|mbc07]] ([[User talk:mbc07|talk]]) 00:16, 5 December 2015 (CET)
:::Err, ping? - [[User:mbc07|mbc07]] ([[User talk:mbc07|talk]]) 18:14, 6 December 2015 (CET)


OK, now have this in place [[Template:Testing/entry/sandbox/testcases|in the sandbox]]. Please let me know your thoughts. [[User:Kolano|Kolano]] ([[User talk:Kolano|talk]]) 02:02, 21 November 2015 (CET)
::::I'm not sure this will work out. Try using the "Collapse" button, it seems to cause the full table to disappear now. Didn't investigate further to try to see why. Perhaps you just missed pulling in one of the relevant classes here, but I'm not sure. [[User:Kolano|Kolano]] ([[User talk:Kolano|talk]]) 18:54, 6 December 2015 (CET)


: The concept is very clever, I really like it, but I feel the current implementation, while working as intended, could be a little simpler:
:::::When implemented it'll work fine without breaking any functionality we currently have, this table from the example I put above isn't actually calling {{tl|Testing/entry}} <s>and I was too lazy to actually check and include all required classes</s> (fixed, should mimic {{tl|Testing}} correctly now). I want to hear about possible points you may have in favour/against of this approach before moving further (and if possible, address possible complaints)... - [[User:mbc07|mbc07]] ([[User talk:mbc07|talk]]) 19:23, 6 December 2015 (CET)
:* Check if <nowiki>{{VersionRevision|{{{revision|}}}}}</nowiki> is bigger or equal than <nowiki>{{#var:tl_testing_entry_lastrev|0}}</nowiki> (will automatically return 0 if it's the first entry).
::*If true, set tl_testing_entry_lastrev to <nowiki>{{VersionRevision|{{{revision|}}}}}</nowiki>
::*If false, tag the page with the unordered category (and possibly set a flag to avoid reincluding the category again if already included, similar to {{tl|s}}).
: The whole logic can be done in a single <nowiki>{{#ifexpr}}</nowiki> that way! But independent of that, it looks really good to me, +1 for merging it in the main template! - [[User:Jhonn|Jhonn]] ([[User talk:Jhonn|talk]])


The problem I had with that is that the #expr logic doesn't treat blank string as 0 and errors on ("" - #). So the variable needs to be initialized to 0 for things to work out or some conversion of "" to 0 performed. The pre-init in testing\start seemed like the least work, and also helps avoid things being non-initialized if a page includes no test entries.
::::::I was just worried we'd break the collapse when using colspans somehow. Still seems to be funny if the sort buttons are used, so we may need to watch out for breakage if implemented. Only other concern I have is that we then label the Device column as CPU\GPU, but that's probably not the end of the world. [[User:Kolano|Kolano]] ([[User talk:Kolano|talk]]) 02:13, 7 December 2015 (CET)
: That's what the additional parameter in #var does (<nowiki>{{#var:tl_testing_entry_lastrev|0}}</nowiki> would return 0 if the variable is not set or empty, or whatever else you put in the parameter -- another example is in OS category, we save the regexp value in a variable, if no suitable string is found, the variable will be empty but will return "Linux" instead, because of the additional parameter). - [[User:Jhonn|Jhonn]] ([[User talk:Jhonn|talk]]) 05:56, 22 November 2015 (CET)
:: OK, will try to revise later today. [[User:Kolano|Kolano]] ([[User talk:Kolano|talk]]) 17:10, 22 November 2015 (CET)
::: Implemented, will take a while to parse through all the pages though. [[User:Kolano|Kolano]] ([[User talk:Kolano|talk]]) 18:06, 22 November 2015 (CET)
Is there a good reason to worry about a category being included on a page multiple times? Such is invisible and it seems like more work to check before including each. [[User:Kolano|Kolano]] ([[User talk:Kolano|talk]]) 19:21, 21 November 2015 (CET)
: I was thinking of reducing possible overhead (a somewhat working example is on {{tl|s}}). However, I never researched if it really reduces the overhead or not. In that case we can just scrap the checks (in {{tl|s}} too)... - [[User:Jhonn|Jhonn]] ([[User talk:Jhonn|talk]]) 05:56, 22 November 2015 (CET)


OK, I'm seeing a few issues in the wider deployment:
:::::::Nice catch, I'll try to do something regarding the sort buttons. And about CPU/GPU, well, in that approach we in fact do not name the column as "device" but I think it's intuitive enough to the reader (e.g. a device of course will have a CPU/GPU so in the end it makes sense using colspan here). - [[User:mbc07|mbc07]] ([[User talk:mbc07|talk]]) 03:31, 7 December 2015 (CET)
*Some pages with a single test show as out of order, unclear why
*Ishura(sp)? builds are not handled correctly resulting in in page error messages
*Something may be wrong with pages names that include parenthesis, seems odd but many of the out of order flagged pages include them even though there tests are in order

Revision as of 21:56, 6 August 2017

Handling of "User" Entry

The "User" entry needs a correction. The "Special:Contributions" page won't exist for users that aren't registered, and it's only displayed for such. I would lean to using the "Special:Contributions" page for users that exist and a generic search on the user name otherwise.Kolano 09:02, 17 February 2012 (CET)

Actually, on closer look, the "Special:Contributions" does work for registered users that don't have user pages. Not clear if there is a way to identify unregistered users and provide a generic search for them instead.Kolano 09:04, 17 February 2012 (CET)

Android Testing Results

I recently saw some users posting test results of Dolphin for Android in the forums. Should we start including them in the wiki? mbc07 (talk)

We can, I may need to make some template edits to handle categories, documentation updates, and etc for such. Kolano (talk) 08:28, 1 May 2014 (CEST)

The main problem I see is that... will users even know what's in their phones? Sure they will possibly know what their phone is, so I guess that gives us something to work from, but are you ok with getting CPU and GPU information from a phone name? And how to phone names fit in the template? - MayImilae (talk) 16:30, 5 May 2014 (CEST)

Well, if the user want to upload an Android test then we should require they at least know what CPU and GPU their phone have. Another possible (but a little ugly) solution would be matching CPU and GPU from a table (maybe in a sub template) with phone names to do the thing, so, in the test entry the user would just put something like this:

{{testing/entry|revision=4.0-xxx|OS=Android 4.3|phone=LG G2|result=Black screen, buggy drivers|tester=AndroidUser}}

and in the page it would be replaced by the common template (Revision=4.0-xxx, OS=Android 4.3, CPU=Krait 400 Quad Core @ 2.26GHz, GPU=Adreno 330). The main issue with this approach would be the big nightmares we would get, like maintaining the list updated with all of Android phones available, not to mention having a lot of entries for the same phone in the sub template table (Samsung Galaxies for example, flagship models get at least a Qualcomm variant and an Exynos variant in most cases) mbc07 (talk)


(quoting from Linux Purge discussion)

It looks like you did capture some of the mobile CPU/GPU stuff. This also seems fine, though I also have a bad feeling many folks will have no idea what CPU/GPU are in their phones and we'll be cleaning stuff up a lot. I guess that would be dependent on actually getting some Android test results though. Kolano (talk) 15:24, 17 November 2015 (CET)

I won't work on this for a while, just realized some other issues we'll certainly have in current design (e.g. two different devices may use exactly the same CPU/GPU and Android version and yet may perform completely different because of OEM shipping older/newer GPU drivers or one including optimizations not available in the other). Including device model would fix that but it would also probably require some redesigns. In other words, at least currently it's not worth the work needed, we still don't have any Android testing entry (on the wiki)... - mbc07 (talk)
We now have one, from a video of 4 Swords on Android, but that's from ~2.5 years ago. Kolano (talk) 13:41, 2 December 2015 (CET)
I think I found a way to account for Android test entries, what about adding a new device= parameter to {{Testing/entry}} that get the device model (in the common format <brand> <series> <model> and if necessary <variant> -- but only if that specific variant ships with a different SoC)? When specifying a device this entry would span across CPU and GPU columns (to prevent breaking current table), solving the issue of different phones with the same SoC performing different while maintaining simple for the user (which will need to know only his phone model). Also, a tech-savvy reader could quickly search for the phone model if he wants to know that CPU/GPU that specific device has. I didn't code a sandbox but that's what I mean:
Test Entries
Revision OS Version CPU GPU Result Tester
4.0-3423 Windows 10 Intel Core i7-3630QM @ 2.4GHz NVIDIA GeForce GT 635M Great mbc07
4.0-4636 Android 4.4.2 Samsung Galaxy S4 (I9506) Good SammyDroid
4.0-7423 Windows 8.1 Intel Core i5-5200U @ 2.2GHz NVIDIA GeForce GT 920M Great BroadwellPC
4.0-8031 Android 5.0.2 Sony Xperia Z3 Plus Good Xperia007
4.0-8031 Android 5.1 Motorola Moto X Play Good MotoDroid
It would also simplify category generation (we would need to handle only device brand instead of mobile CPU/GPU). So, any thoughts? - mbc07 (talk) 00:16, 5 December 2015 (CET)
Err, ping? - mbc07 (talk) 18:14, 6 December 2015 (CET)
I'm not sure this will work out. Try using the "Collapse" button, it seems to cause the full table to disappear now. Didn't investigate further to try to see why. Perhaps you just missed pulling in one of the relevant classes here, but I'm not sure. Kolano (talk) 18:54, 6 December 2015 (CET)
When implemented it'll work fine without breaking any functionality we currently have, this table from the example I put above isn't actually calling {{Testing/entry}} and I was too lazy to actually check and include all required classes (fixed, should mimic {{Testing}} correctly now). I want to hear about possible points you may have in favour/against of this approach before moving further (and if possible, address possible complaints)... - mbc07 (talk) 19:23, 6 December 2015 (CET)
I was just worried we'd break the collapse when using colspans somehow. Still seems to be funny if the sort buttons are used, so we may need to watch out for breakage if implemented. Only other concern I have is that we then label the Device column as CPU\GPU, but that's probably not the end of the world. Kolano (talk) 02:13, 7 December 2015 (CET)
Nice catch, I'll try to do something regarding the sort buttons. And about CPU/GPU, well, in that approach we in fact do not name the column as "device" but I think it's intuitive enough to the reader (e.g. a device of course will have a CPU/GPU so in the end it makes sense using colspan here). - mbc07 (talk) 03:31, 7 December 2015 (CET)