User talk:Dr ishmael/Gw.dat

Just wanted to let you know when I clicked your link for GWUnpacker, my Avast Antivirus started screaming. It could be a false positive (had a couple of those recently) but I'm not risking it. I know it probably isn't your intention to harm anyone, but I'd look into it if I were you :) Lยкץ๒๏ץ talk  19:36, November 25, 2009 (UTC)


 * Norton IS didn't find it important enough to warrant a word of caution. --- [[Image:VipermagiSig.JPG]] -- (contribs) &emsp;(talk)  19:54, November 25, 2009 (UTC)


 * I can assure you, it's a false positive. &mdash;Dr Ishmael [[Image:Diablo_the_chicken.gif]] 19:55, November 25, 2009 (UTC)


 * Warning: link to GW-Unpacker is marked as Virus Malware - win32. trojan

http://www.xentax.com/downloads/tools/hosted/boyc/GW_Unpacker.zip


 * It's a false positive due to something in ATEXreader.exe. It's not actually a Trojan.  &mdash;Dr Ishmael Diablo_the_chicken.gif 18:18, 19 June 2011 (UTC)

String Files
The data you have here is interesting. I've been looking into the string stuff recently too. Did you ever uncover any additional information on them? 89.108.110.170 03:53, 2 February 2012 (UTC)


 * Unfortunately, no - I still can't make any sense of the non-UTF data. By watching when the empty strings in the last file become populated, though, I'm pretty certain that they contain the text for quest dialogue, at the least.  I guess they decided to compress those strings in some way because quest dialogue would always be longer than skill descriptions?  I dunno.  If someone could just figure out how to read them, though, that would make writing up new quest articles a snap.  &mdash;Dr Ishmael Diablo_the_chicken.gif 05:12, 2 February 2012 (UTC)


 * I've researched how the engine uses them internally and they definitely contain everything--all of the quest dialogue, item names, etc. The non-UTF data is not compression, it's their string encryption. Every encrypted string requires a unique key to be decrypted. It seems the only hope would be if someone cracked their algorithm. 89.108.110.170 07:41, 2 February 2012 (UTC)


 * Encryption, huh? I hadn't considered that.  Are you sure it's 1 unique key for every string, or are there a set of keys that are reused?  I wonder if that might be the purpose of Byte 2 or Byte 4, identifying which key the string was encrypted with.  If so, then I wonder if it would be possible to grab the keys out of memory while the game is decrypting.
 * Now I wish I knew more about using debuggers and Assembly. &mdash;Dr Ishmael Diablo_the_chicken.gif 15:28, 2 February 2012 (UTC)


 * Out of all of the keys that I currently know, none of them are the same. It doesn't seem likely that there is a set of them. Your theory on byte 2 being the lowest-value non-punctuation character within the description fits every encrypted string I've checked perfectly, but so far I haven't found any correlation between this and the key. It seems byte 4 is usually 7 (occasionally 6) and 5 for strings that are only 2 bytes. I'm still researching the difference between strings with 6 or 7. 89.108.110.170 18:43, 2 February 2012 (UTC)


 * In that case, the keys are very likely stored somewhere within Gw.dat themselves, since it seems unlikely that all ~60k keys are compiled into the .exe. Although, I suppose it's possible that they use symmetric-key encryption and they generate the keys on-the-fly somehow, using the file/string ID as a parameter.
 * What do these keys look like? How long are they?  Could you post some examples of key-string pairs that you've found?  &mdash;Dr Ishmael Diablo_the_chicken.gif 19:26, 2 February 2012 (UTC)


 * The exe references text encoding/decoding and contains log messages with "encoding encrypted string" and "check the string's security field" so it seems that the client is capable of encoding and how the key is generated can be controlled. It's worth noting that the same key is used for all languages even though the encrypted data for each language differs. Example:
 * 89.108.110.170 07:23, 3 February 2012 (UTC)
 * 89.108.110.170 07:23, 3 February 2012 (UTC)


 * Only 48-bit keys? That's not as bad as I was expecting.  And having language-independent keys fits with my idea that the key generation is controlled by the file/string ID, since that is also language-independent.
 * I've tried running that example through a few symmetric algorithms (Blowfish, RC4, RC5, and CAST5), both encrypting and decrypting, but I can't get any of them produce the expected output. The key length automatically rules out others, like Twofish and AES/Rijndael, that can't use 48-bit keys.
 * I'll have to search the text files for those encrypted strings tonight, see if I can find any more clues there. &mdash;Dr Ishmael Diablo_the_chicken.gif 17:40, 3 February 2012 (UTC)


 * Well, no clue in the text data files - found the encrypted string, and that's all it is: .  It's located at file 8, string 169 (A9).  I did see that the next 2 strings also have b2 = 42, with string lengths of 10 and 3.  With this having length of 7, I bet those two are   and  .  Not that that will really help us much, unfortunately.  &mdash;Dr Ishmael Diablo_the_chicken.gif 23:35, 3 February 2012 (UTC)

Yeah, the next two are in fact Belt Pouch and Bag and yep they're in file 8. I output all ~71K encrypted strings to one file in order (the engine assigns them all Ids internally) so I can easily look at the data. Haven't found anything further yet, but I'll let you know if I do. 89.108.110.170 03:24, 4 February 2012 (UTC)


 * Would you mind giving me a quick rundown of what you're doing to examine the exe? Memory debugger, decompiler, voodoo magic?  I don't know a whole lot about that, and I'd like to learn.  &mdash;Dr Ishmael Diablo_the_chicken.gif 04:21, 4 February 2012 (UTC)


 * Well for one I keep copies of the exes after every build and compare them. It's not too difficult to pick out added data. The hard coded skill data, for example. Every skill has a 160 byte block. They each have three string IDs specified (name, description, consise desc), the hash for the icon, skill type, etc. Not gonna go into anything too advanced in the open, for obvious reasons. 89.108.110.170 07:38, 4 February 2012 (UTC)


 * We can take this to email, then. I'm dr.ishmael on the gmail.  &mdash;Dr Ishmael Diablo_the_chicken.gif 17:11, 4 February 2012 (UTC)
 * Haven't heard from you yet, you been busy the past few days? &mdash;Dr Ishmael Diablo_the_chicken.gif 17:18, 7 February 2012 (UTC)


 * So far I haven't been able to decipher how the engine is using the key and coming up with the text so not much more to report right now, but am semi interested in WoC tomorrow. Nine new quests means lots more encrypted data to analyze. :P Not to mention whatever else they throw in the mix. That "maintenance" build the other day increased the size of the exe by like 12Kb, which is something that hasn't happened in a while, and it threw off my offsets a little. Seems a little odd, not sure if WoC related or what. 89.108.110.170 01:13, 9 February 2012 (UTC)


 * Well, I realized something as I was falling asleep last night. It's probably not very useful, but I realized that I was wrong to think about bytes 2 and 3 separately.  The strings are in Unicode, right?  So why was I thinking of the lowest byte-value character as being ASCII?  Duh, it's 2 and 3 together that do this.  Like I said, not very useful.
 * Anyway, I'd still like to know how to do what you're doing, so if you wouldn't mind contacting me by email, I'd appreciate it. &mdash;Dr Ishmael Diablo_the_chicken.gif 16:29, 9 February 2012 (UTC)


 * Ah, yes that does make sense. Presumably 4 and 5 follow the same. Wasn't really expecting ~1500 new strings today, they filled 96 quick. Plenty to rummage through. I'll try and mail you ASAP, although I'm not sure what all you'd like to know. 89.108.110.170 00:00, 10 February 2012 (UTC)