User talk:Xerxes

ID Guide
So I've had this ID guide sitting on my user page for forever and I'm kind of sick of looking at it. I rewrote the entire thing to remove first/second person, did a few grammatical checks, swapped some stuff around, and I really think it's in a presentable state. I'd like to get some review on it if possible for a move to its own page like GameID Confirmation Guide or something that can be linked into Category:Tutorials. Any thoughts would be appreciated. - Xerxes (talk) 09:58, 25 November 2017 (CET)

I moved it to User:Xerxes/GameID confirmation guide but I'll be sad if it's fated to stay in this hidden user subpage forever... - Xerxes (talk) 12:11, 2 December 2017 (CET)


 * Ok...

(talk) 03:43, 3 December 2017 (CET)
 * I'd like to see things like R8XE52 / R8XZ52 resolved, and purged from the discussion.
 * Something seems wrong with our second layer bullets (i.e. weird double bullets). I cleaned it up with indents.
 * Under "How does the wiki handle IDs" should we cover the cases with XYZ titles differentiated by languages supported.
 * The sources should likely be re-ordered, migrating the sources with partial IDs to the end of the list.

Gamecube
These have all been investigated as of right now (again). - Xerxes (talk) 21:04, 14 June 2017 (CEST)

U Region GameIDs
These have all been investigated as of right now. The only ones skipped were the demo discs as that's its own nightmare for another time. It turns out they WERE all Australian releases though, besides one. - Xerxes (talk) 13:20, 9 June 2017 (CEST)

V Region GameIDs
These were all investigated, but I only found a cover for Skylanders: Trap Team and Epic Mickey 2: The Power of Two. - Xerxes (talk) 11:00, 12 June 2017 (CEST)

H Region GameIDs
These have all been investigated. They were entirely Netherlands region releases. - Xerxes (talk) 19:41, 19 June 2017 (CEST)

W Region GameIDs
This region ID is used for Chinese language releases of games when available. Since video game consoles were banned in China from 2000-2015, this means that the titles could only have been released in Hong Kong, Macau, Singapore, or Taiwan realistically. Though a title may see release in multiple countries from this list, it seems that the ID prefix stays the same and the W region is preserved. For example, Super Mario Galaxy 2 had two releases with the W region code, one for Taiwan and the other for Hong Kong/Macau (note in particular the English text right above the ID stating this).

Now, to complicated the matter, the Hong Kong/Macau releases of 1st party titles aren't published by Nintendo, they're published by iQue. iQue nowadays is a 100% subsidiary of Nintendo and effectively functions as their Chinese branch, but before 2013 they were actually a joint partnership between Nintendo and an independent Chinese entrepreneur to attempt to bypass China's harsh video game restrictions. The problem here is that, at the time that these titles saw release, this joint ownership still existed; this both sheds doubt on the use of 01 publisher code for these IDs as publishing wasn't done exclusively by Nintendo or with the Nintendo brand, and iQue also creates a problem with the infobox since no Hong Kong region exists as far as I am aware (and it seems rather specific to make just for these fringe cases). - Xerxes (talk) 08:19, 21 June 2017 (CEST)


 * These have all been investigated. RLWW78 is the only full release that isn't Chinese, if GameTDB is to be believed, and D43W01 was skipped for being a bonus disc. - Xerxes (talk) 09:14, 23 June 2017 (CEST)

Publisher ID 28
ID 28 is the JP publisher ID used currently for all four Gamecube Kemco-published titles on the wiki: Egg Mania: Eggstreme Madness, Batman: Dark Tomorrow, Universal Studios Theme Parks Adventure, and Rogue Ops. Sometimes publishers just use different codes in Japan for some reason, and I can buy that, but these IDs are particularly mysterious to me because I can't confirm them, no GameTDB listing at all. They do exist though, they have JP covers and they have redump entries, but since redump uses the four character codes and not the six character I can't check the publisher ID. I trust the IDs that are here in any case because User:DanbSky's contributions from a quick survey are usually correct and sourced, and seem to come from his own collection, which would explain why they don't exist anywhere else (first time they've been documented). But it's probably worth mentioning that I have no way to check if they're correct or not short of tracking down the disc myself. - Xerxes (talk) 12:44, 16 December 2016 (CET)

EA
^is this EA's fault or ours? I'm assuming the J69 are mistakes. - Xerxes (talk) 03:22, 4 June 2017 (CEST)


 * Mostly seems to be our fault, where it was presumed the JP publisher ID would match up with other regions. I cleaned up some of this, but there seem to be at least a few titles I'm having trouble confirming. A few title may actually use the "69" ID. For instance:
 * NBA Street: http://www.gametdb.com/Wii/GNSJ69
 * SSX Tricky: http://www.gametdb.com/Wii/GSTJ69
 * Harry Potter and the Chamber of Secrets: http://www.gametdb.com/Wii/GHSJ69
 * Need for Speed: Most Wanted: http://www.gametdb.com/Wii/GOWJ69 (we list GOWJ13 for this too, but I can't confirm it)
 * SSX on Tour: http://www.gametdb.com/Wii/GXOJ69
 * I'm guessing these may be titles developed by EA studios outside of JP, but that's unclear. Kolano (talk) 20:15, 4 June 2017 (CEST)

The problem is that this is the exact kind of edge case where GameTDB tends to have mistakes, but there's also no other source to double check this against to confirm or deny that. I don't think we'll ever get a straight answer unless someone has the JP releases of these games, dumps them, then checks the ID and updates them here. The worst part is that they may actually be right and so nobody who has these dumped would have a reason to change the IDs, in which case we'd never know for sure. In terms of the wiki, I think it comes down to a decision whether or not to trust GameTDB or assume they're mistakes and make them consistent, but since GameTDB is the most reliable source there's not really much choice here. If you have a more creative solution, I'd be interested in hearing it, because I'm stumped.

Well, I came up with a creative idea. Doesn't Dolphin collect usage statistics now? And haven't those statistics included GameIDs since the start? Could that be used to see if some of these IDs exist or not, just by ctrl-F for these IDs? I mean if nobody's played it nobody's played it, but that's the best I got. - Xerxes (talk) 03:41, 5 June 2017 (CEST)


 * K, not sure on this.


 * Regarding the category, it may be helpful, but the capture of details, as here, should help avoid repeated researching which is probably preferable.


 * I'll also try to see if we can get access to the GameID inventory the dev's maintain from data scraped from Dolphin to see if we can confirm things that way (though there are many bad IDs in that list from custom generated ISOs we may need to be careful of). Kolano (talk) 22:57, 7 June 2017 (CEST)

Well of course there will be homebrew invalid IDs in there, I don't doubt that. I'm not suggesting to just mass add every ID from the usage statistics blindly. My master plan here is to use the power of statistics to confirm IDs. The thing is, because IDs are so different between games, if you know exactly what ID you're looking for, there shouldn't be any confusion. In the EA example above, I highly doubt there would be a homebrew title with the exact same GameID as an EA title, but with J69 suffix instead of a J13 suffix. That's the perfect situation where this would be useful. And of course, validity can be weighted by the amount of unique players who have played a certain ID. If there's only 1 record of it being played, maybe its a little tenuous whether it's legit or not, but if there's 10 or 20, then I think that stands on its own as confirmation.

Using this method for confirmation, since there's literally no other source to confirm Wii games with sketchy GameTDB entries and no cover scans, may actually be the only way to figure out whether these IDs are legitimate or not. This would make the talk page entries pointless since confirmation by hand has already been proven to be near impossible for these titles. I still advocate confirmation by hand when possible and I will continue to do so, as it sheds light on what IDs are possible or not, and I will still fill out the talk pages per your request, but maybe in the future instead of talk pages which historically haven't been particularly effective (nobody's fault), the category would be more practical. It would be one big list of raw IDs to confirm against the usage statistics data, making that process much smoother. What's the alternative? Just open every talk page until you find an Unconfirmed ID section, or use some function to search for that title on talk pages? I didn't even standardize that title until a few days ago, so even without the category I'd still have to go back and retitle all my talk page entries to automate this.

In any case the category actually has its own added benefits; it lets us use wiki tools like DPL to study IDs and their trends, while also being able to remove sketchy IDs from the results and be reasonably certain that the IDs looked at are legitimate. The problem here of course is this wouldn't be 100% useful unless every single confirmed ID had its own separate category of Confirmed IDs as well, but that would be a massive amount of work for little reward, whereas the Unconfirmed IDs category is much easier and, with enough data, functions effectively the same. - Xerxes (talk) 09:20, 8 June 2017 (CEST)


 * I don't think we'd want to have unconfirmed ID hanging around, so speedy deletion would be preferable, and it can always be recreated later if needs to. You can even open a discussion regarding the still unconfirmed ID in title's talk page if you'd like. The recently deleted ID will re-appear in list of missing ID so you can find out how popular is that "unconfirmed" ID. Again, speedy deletion is preferable because who knows if it's already ranked low (or won't appear) in the list of missing ID. Lucario (talk) 11:05, 11 June 2017 (CEST)


 * Oh, this functionality already exists? That's awesome. So if I understand this right, when there's a missing ID on the wiki, but it's in usage statistics, this page updates and adds the ID along with showing the all time playcount from the gathered data; or, if it wasn't played, it doesn't appear on the list. Is that right? My only fear with deleting unconfirmed IDs is actually one of degrees. For example, with Epic Mickey 2, the K region ID I'm almost certain exists because there's Korean covers, but I can't find a back scan to be 100% sure that's the right ID; however, the F region ID I'm pretty sure doesn't exist from the evidence I've found. If for example we were to delete both as unconfirmed, yes we can add the K region right back afterwards, but this would actually break the right click wiki functionality in Dolphin for the small subset of Korean Dolphin users. Maybe not a big deal for one game, but I've found a LOT of unsubstantiated IDs, and if we were to go through and start deleting them and seeing what happens I'm afraid of the end user impact of that. That's one reason I opted for a slower approach with a category. - Xerxes (talk) 11:50, 11 June 2017 (CEST)
 * The missing-gameid.txt I later found out only registers when people follow an ID link from Dolphin and land on a page creation (or at least I think I remember that? I can't find it again...), which was the exact situation I was worried about having happen. So, that's even more volatile than I thought. Basically if we use that tool in the way you suggested, we'd be sitting back and deliberately waiting for users to have functionality broken for them, so it'll register on the list. That does not seem like good design. - Xerxes (talk) 13:59, 19 August 2017 (CEST)

JALT01
From User talk:PowerKitten: Please substantiate this ID. GameFAQs and GameTDB both give JALK01, and the wiki has zero other IDs using a T region code besides that one (you can see a full automatic list below, which contains only JALT01).

- Xerxes (talk) 21:43, 4 October 2017 (CEST)

I'm replying to your post on my user talk here since I don't think you'll be notified if I respond there.

I submit these Nintendo Update Server TMD links for evidence. These are all four Virtual Console versions of Contra III: The Alien Wars that were released. JALE01, JALJ01, JALP01, and JALT01

These are on the official servers and are properly signed by Nintendo. JALK01 however, is not and causes the update server to 404. I'm currently investigating the correct GameIDs by scraping the Wii update server, and I've discovered that the "K" RegionID which is wildly assumed to be correct for all South Korean releases simply isn't correct. From my investigation, the "T" and "Q" RegionIDs are more commonly used for South Korean online releases. So far, the only online GameIDs using the K RegionID I've been able to verify are HCRK01, HCFK01, and HAVK01. HCRK01 on NUS for verification purposes. I'll probably end up doing a full write-up on how widespread this incorrect assumption on South Korean GameIDs was when I'm done scraping the NUS server to get a verifiable list of the valid IDs for downloadable software. --PowerKitten (talk) 00:00, 5 October 2017 (CEST)


 * That's a great idea. I never knew you could just ping Nintendo's servers like that to confirm IDs. I actually didn't think there was a good first party source for the digital GameIDs, I've mainly focused on disc-based titles because they have prefixes printed on the covers. Does this work for WiiWare games as well? How do the urls correspond to the IDs? I have a lot of questions actually... Maybe I'll hold back a bit. I'm very interested in this. - Xerxes (talk) 00:25, 5 October 2017 (CEST)


 * Don't be afraid to ask what you want to know. Worst that can happen is that I don't know the answer. This method works for anything you can download from the Wii Shop Channel, including Wii Channels and WiiWare games. The URLs correspond to the GameIDs rather easily. I'll use HCRK01 as an example. The "00010001" part means it's Wii Shop content, and the "4843524B" part is simply the hexadecimal representation of the GameID without the Publisher Code. It decodes to "HCRK" as you would expect. Getting the Publisher Code is slightly more difficult and requires opening the TMD file with a hex editor. (TMD stands for "Title MetaData" and provides information on an online title like the GameID and the version number.) Here's a screenshot of the TMD in a Hex Editor with the relevant portion highlighted. The "01" there is the publisher code, and it can be consistently found there in my experience. --PowerKitten (talk) 00:49, 5 October 2017 (CEST)

So you're telling me that every digital Wii game has publicly available title information sorted in this exact consistent format across all regions? What kind of tools are you using for the data scraping? Are you comparing against a database (like ours) or doing it manually or what? I've never really trusted GameTDB, so being able to stop relying on them for IDs would be awesome if it can be done. - Xerxes (talk) 01:15, 5 October 2017 (CEST)


 * Yep. Every digital Wii game has publicly avaliable information. Even the ones that Nintendo's removed from the Wii Shop are still up there. I'm using a simple CURL command with some basic assumptions. I'm assuming that System Codes and Region Codes are only letters, and that all of the GameIDs for Wii Shop content can only use uppercase letters or digits. These assumptions were based on WiiBrew's title database and mostly just done to condense the search space so I wouldn't be waiting around all day for the scrape to finish.

curl -f http://ccs.cdn.shop.wii.com/ccs/download/00010001{41,42,43,44,45,46,47,48,49,4A,4B,4C,4D,4E,4F,50,51,52,53,54,55,56,57,58,59,5A}{30,31,32,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,4A,4B,4C,4D,4E,4F,50,51,52,53,54,55,56,57,58,59,5A}{30,31,32,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,4A,4B,4C,4D,4E,4F,50,51,52,53,54,55,56,57,58,59,5A}{41,42,43,44,45,46,47,48,49,4A,4B,4C,4D,4E,4F,50,51,52,53,54,55,56,57,58,59,5A}/tmd --create-dirs --output ..\..\TMDs\00010001#1#2#3#4.tmd

This is the command I used. It only saves a TMD file if there actually is one, which is important because otherwise, it would save the 404 error HTML page as every wrong GameID which would make verification a hassle. My scrape should be done in a couple hours so we'll be able to take full advantage of it. If I was to throw away all assumptions as to what GameIDs are allowed I'd end up with a search space 4,294,967,296 items large which might take until 2019 to fully scrape and would be extremely unlikely to actually find anything more. - PowerKitten (talk) 01:38, 5 October 2017 (CEST)


 * Those are good assumptions to make, it lines up with what we have documented about ID structure on GameIDs. You might want to clean that page up when you've done some analysis. (If you're really really curious, only one ID on the wiki uses a non-uppercase letter and it's GNHE5d, and that's GameCube anyways.) I'm guessing even though this is publicly available, it's probably still not ok to transfer the files around through intermediates as a big archive, so I guess I'll have to run that as well to get my own copy to mess with. I'm really not a lawyer though, but I'd like to err on the side of caution. - Xerxes (talk) 02:10, 5 October 2017 (CEST)


 * I think it should be okay to transfer the set around. You can use the TMDs to download encrypted game content with NUS Downloader, but you're still going to need a valid ticket to actually decrypt games and use them in Dolphin, for example. You can only get tickets by actually buying the game on the Wii Shop Channel. I doubt Nintendo's going to pursue it, and Nintendo would probably prefer people downloading them from elsewhere to people using up Nintendo's server time and bandwidth scraping for TMDs. In the meantime, I'll work on creating a list of my verified GameIDs on my User Page. - PowerKitten (talk) 02:26, 5 October 2017 (CEST)

Something else you may want to consider. There was once an effort a few years ago to populate GameIDs here based on the played game statistics that Dolphin automatically collects. I'm sure that those scripts still exist somewhere, so maybe they can be adapted and reused in this situation for mass addition of scraped IDs? I wasn't involved in that process at all, but it seems like a more reasonable approach than hand-editing in each ID individually. The places I'd ask about it would be User talk:mbc07 since he's been a long time administrator, or on IRC channel #dolphin-emu @ irc.freenode.net. I'm sure if you explain what you're trying to do and your methodology on IRC someone should point you in the right direction. - Xerxes (talk) 13:26, 5 October 2017 (CEST)


 * AFAIK we used Redump.org and GameTDB info to populate the game IDs we had at that time on the wiki. However, I never saw the actual script which collected that info, for those details I would ask either delroth or Parlane (but it's probably easier to talk to them on the IRC, they rarely visits the wiki) - mbc07 (talk) 02:06, 8 October 2017 (CEST)

Yeah oops, I think I got my time tables a bit mixed up (user statistics was after the wiki IDs if I remember correctly). I'll probably try forum PMs since I don't do too well on IRC historically. But I'm going to wait until there's something substantial and a game plan in place first before dragging too many people into this. - Xerxes (talk) 17:59, 8 October 2017 (CEST)

So I've been reading about TMDs on WiiBrew a fair bit and what I can't find is how to get the name of the game the ID is for from the metadata. There seems to be another step that needs to be done. Having a raw list of all IDs would be great, but they can't be mass added unless we also have a game title to redirect to for each one. I've been mulling over a few ideas. First of all, theoretically we could just make them all broken redirects so they'd show up on Special:BrokenRedirects, but that doesn't seem very efficient. Another idea would be to create from the TMD archive an alphabetized index, and then compare the index against one or multiple existing databases looking for matching first three characters to get the titles, then that'd leave an index with titled IDs in a similar format to GameTDB's .txt index that could be used for a mass addition script (but then we're still relying on third party databases). If there's some really easy way here to get the game title from the NUS once you have the ID, that would be the best solution. - Xerxes (talk) 18:07, 9 October 2017 (CEST)


 * I can get the game titles from the Wii Shop Channel once I have the ID, but that has to be done manually. - PowerKitten (talk) 19:34, 9 October 2017 (CEST)