GuildWiki talk:Interactive map project

Java vs. Javascript

I guess the first few steps would involve deciding which approach to use.

A full fledged mashup which would overlay data on top of an image. This could actually be accomplished with PHP / GD. Map coordinates and an appropriate icon / note can be pulled from a DB and rendered on top of the image.

Or if we wanted to take an approach that would be more of a tool to create an image that is saved. The mashup would be more versatile and allow for more flexibility and options in what can be done.

The various maps located at http://broadbandreports.com is a very good example of a mashup. All the data is housed locally and they basically plot it in real time on top of a Google Map.

The first approach would require a separate infrastructure to support the maps. Map "bases" would have to be installed (continents, zones, etc.).

Some of the obvious benefits of a system that would accomplish this is, Higher Quality and Resolution of mission maps, boss locations etc. Currently we are forced to view the information in whatever resolution the user decided to capture the screenshot. Plotting this information on top of a pre-existing high quality map would be a superior alternative to this.

Additionally data can be 'tweaked' without the need to generate an entirely new screenshot.

I would be happy to design the database structure that would be required to implement a PHP / Java / MySQL variant.

Ideally, a Java applet would be created which would allow panning, zoom, etc of the maps. Also a frontend would have to be created to get POIs (Points of Interest) into the actual database for the mashup to pull from.

I can also provide a development platform for the project. ErkDog 15:41, 20 November 2006 (CST)

I have some experience developing Java applets, so I would be happy to work on that side of things. I'm pretty busy for the next two weeks, but after that I have a bit of a break when I could work on this. - Lord Ehzed 15:45, 20 November 2006 (CST)

Previous work

I've already posted this link in GuildWiki_talk:Community Portal#Mashups_.2F_Maps:

http://www.phototraffic.com/

It's pretty close to what I envisioned for myself, even though closer integration with Guild Wiki would be great, of course. If we still decide to do our own implementation, I'm strongly in favor of a Javascript solution instead of actual client side Java. DeepSearch 03:11, 22 November 2006 (CST)

I think it would be pretty hard to accomplish the things the map client would need to do with JS instead of Java. The map @ pt.com is pretty.... weak.... compared to what I was thinking. It only zooms like two levels making it hard to distinguish POIs close together. It also has nothing on it other than Elites, no other POIs or routes. But it is definitely a proof of concept that we can do what we want to do with the Google API and custom maps. Our implementation would require one full map for each continent for elites, then perhaps additional maps for missions to show routes. The zoomed in POI for an elite can be shown in a map window on the page for that elite. Clicking on a link under the map would switch you to the larger map which is more easily navigable. ErkDog 08:08, 22 November 2006 (CST)

I don't think anything you mentioned isn't doable with Google Maps and given the choice, I would definitely pick JS over Java. --Fyren 08:49, 22 November 2006 (CST)

As Fyren said, all that is possible with Google Maps. The only thing I'm not a 100% sure of how to implement are routes, but Google can do it, so we should be able to as well. DeepSearch 07:04, 26 November 2006 (CST)

I agree that JS is far better for this kind of thing than Java. Downloading an applet is very inelegant and Google Maps proves that the kind of thing we want to do is possible in JavaScript. <LordBiro>/<Talk> 10:11, 4 December 2006 (CST)

Map data and coordinate system

I played around a bit with the Google Maps API. As it turns out, it's actually extremely simple to use their engine in combination with custom map material. Here's an example I created to get a feel of how the tiles are addressed at different zoom levels:

http://michael.elsdoerfer.name/gwmap/coords

Now the first challenge is going to be to figure out how to fit the Guild Wars maps into there. The following calculations are based on the Elona Map from GWVault:

Each tile has to be 256x256 pixels in size. The number of tiles (in both width and height) is 2^zoomlevel. So the complete map itself is a square as well.

The above Elona map is 6218x5314. Assuming this is the most detailed we can get, it will be used for max zoom level. We need to split it into a number of 256 pixel tiles. 6218/256=24.28, which means we will need 25x25 pieces, which in turn means a 6400x6400 pixel image (again, at max zoom level). 25x25 pieces means we can support 6 zoom levels: 0-5. At zoom level five 2^5=32 tiles are displayed.

So, to create the map data, here's what we have to do:

• Find/create map material we can use. The above elona map seems near perfect (is it based on the U map?), as the outposts, regions and texts seem to be added manually. I guess the dream scenario would be to have the map itself and the "metadata" (outposts etc.) separate, so we could display those as an optional overlay (similar to the Hybrid mode supported by Google Maps). Is anyone familiar with how to create those maps? Maybe it is possible to contact the author of this one.

• Create a script (ImageMagick, Photoshop?) that does the above calculations, creates the necessary zoom levels and splits everything into tiles. See also http://mapki.com/wiki/Automatic_Tile_Cutter

That would be the first step. Secondly, we need a way to address positions on the map. It is possible to use the traditional lat/long system, but it seems to me that it would be simpler to define a custom one. The most straightforward idea of course is to just use pixels. (0/0) to address the top/left corner, (6400/6400) (depending on map) for the right/bottom corner. Is there anything that would speak against this?

Tell me your thoughts. DeepSearch 06:57, 26 November 2006 (CST)

Update: I wrote a small Javascript for photoshop that will run through the process outlined above and create tile images for various zoom levels. You can find it here.
I modified the map above to use the new imagery, and you can see the result here: http://michael.elsdoerfer.name/gwmap/elona/
The next step would be to replace the default MercatorProjection with an Euclidean one, to make adression certain spots easier (although I'm sure it that actually be easier, I don't understand the whole projection thing fully yet). DeepSearch 00:24, 27 November 2006 (CST)

I'd love to help

I've been working on a map using the google maps API that I was planning on just posting on a free host and sharing with my guild etc., but would rather work on a map for guildwiki. So if there's anything I can do to help, I'll do it. Here's my progress: [1] if the source is of any use. I load an xml file for the locations (don't have the outpost up yet) and display them my own GOverlay (because I don't like the default marker class) that I haven't finished. I'm just a hobbyist so the code is a bit sloppy and unfinished. Hopefully someone else is better than me at javascript. But, I can provide raw, no-fog-at-all images, or just make the tiles myself (the tiles I put on the site are low quality due to limit space on free host). just need to know about what quality or size (on the site the image quality was set to 40 (if I remember correctly) using Gimp and all the images take a total of 4mb). So images, gathering data, or writing javascript (if you really want me to and if we are going to use google maps api), I'm here to help :D -Smurf 00:17, 27 November 2006 (CST)

Not bad at all! I see you even have a custom Projection class as well (btw - where is it centered?). Is this your own map material? The missions and towns are done via overlays, if I did understood correctly. But wouldn't those be already on the screenshots from Guild Wars? Did you remove them afterwards? I guess my question is: How is it possible to get a "clean" map, without fog and all the regions visible, but without town/outpost/mission/shrine/trader/text info on it? DeepSearch 00:42, 27 November 2006 (CST)

The center is the top left corner. If your browser allows a website to change the status bar text then it will display the coordinates that your cursor is at. I just use 1 degree lat/lon = 256px. The missions and towns are done with overlays which I'm still unsure if it's the best choice. The map material is my own in the sense that I got it from the game myself and not from a website or someone else, but I didn't use screen shots. -Smurf 01:58, 27 November 2006 (CST)

So, you found out how to pull the map from the hulk of a data file sitting on your computer then? --Thervold 20:07, 27 November 2006 (CST)

I didn't quite understand how you created the maps, but that's probably not a part were I can help with anyway ;) As for the overlays, which obviously have the problem that they are always the same size, no matter the zoom level, which means they will either be very small when zoomed in, or too large when zoomed out. I would suggest to use a similar approach as Google's Hybrid mode. The overlay would be created the same way as the map images itself, split into tiles and would be available in the different zoom levels. The challenge then is how to make cities/outposts still clickable? Maybe it is feasible to intercept clicks on the map and look the position up in a small database if a certain lat/lng is withing a clickable hotspot region. DeepSearch 20:22, 27 November 2006 (CST)

That is an excellent start. Much better than the other site that has been mentioned. As far as the actual data though, we need a way to pop that in and out of a database. I would also think another zoom level or two would be good. As well as higher quality images. Smurf / others, I can provide a space for development. I'll setup an account and a database this evening. ErkDog 08:00, 27 November 2006 (CST)

Actually, that is in interested design question, which we will have to talk about. One of our goals was to have the possibility to integrate the maps into the wiki, e.g. to specify boss locations. How would that we done?
{map boss={PAGENAME}} (how does boss get into the map database?)
{map type=boss lat=4.2 lng=12.5 prof=necromancer name={PAGENAME}} (no need for a db at all? but then how do we display a map with ALL bosses, for example)
At first thought, what make sense to me most, is a two-fold approach: 1) Specify geo locations on boss pages, e.g. {geo lat=4.2 lng=12.5}. 2) Have a bot parse those and add them to a database. 3) To display a location on a boss page, use the boss name to specify what the map has to show: {map boss={PAGENAME}}. If the bot has not yet added that boss the the database, the map will not show (yet).
What are the other options to make the map data editable in wiki-style? DeepSearch 13:05, 27 November 2006 (CST)

The bot approach is probably not the greatest. It would likely be easiest to develop a front end in PHP or Java / JS to handle the modifications of the POI database. http://www.broadbandreports.com is done via the applet itself, so is http://www.frappr.com. Although Frappr uses a Y! API vs Google, but the proof of concept is the same. It can be accomplished with an applet. There would still have to be a custom set of PHP written in order to revert changes. I think the maps would likely be a high target for vandalism. It may be better to simply moderate the changes made to the map POI database. Better to have admins and moderators poke through proposed changes, than to continually deal with cleaning up and rectifying vandalism. ErkDog 13:27, 27 November 2006 (CST)

I see you're point about vandalism, although I'm not sure I agree 100%. If the coordinates are in the wiki in text/code form, and cannot be changed directly on the map, the incentive to manipulate them should be that much higher than the rest. On the contrary, confirming a position can require quite some time, it might be even helpful to have the whole comunity available making sure the data is correct.
I've given what I wrote before a second thought though - I maybe made it more complicated than needed. The map on a boss page really has only one thing to do: Show the location of the boss (and the towns/outposts, if not directly part of the map). All the other data (locations / bosses / skills), that we need a database for, does not need to be on it. So, for maps showing a certain position or route, simply putting the data directly into the wiki map command should be suffice. DeepSearch 20:33, 27 November 2006 (CST)

If there was an easy way to have the API pull data from a wiki page then sure, although I don't know if there is, although, I don't really know how the API works period, lol. But I would think it best to not be pulling data from wiki pages. If the Wiki HTTP/Squid/etc is slow then the maps won't work :( ErkDog 21:35, 27 November 2006 (CST)

I replaced my site [2] with a presear map (nice and small) that zooms in up to 4x and all the images were compressed with Gimp with a quality of 85. The images take up 7.6MB with 1080 tiles in all. So that comes out to 7.2KB per tile on average. Hope this can help in deciding how large we want our map to be. The zoom amount is displayed in the window title. In my opinion 200% is nice and the 400% I can do without. If you want me to put back elona I can but will have to take presear down. -Smurf 22:25, 27 November 2006 (CST)

Yeah, we could probably do without 400%. On the other hand, apart from a little bit of disk space/traffic it doesn't really hurt either, and maybe someone finds it useful. Is this in-game zoom, or also done with Gimp? DeepSearch 06:47, 28 November 2006 (CST)

I used Gimp to resize the map. -Smurf 02:29, 4 December 2006 (CST)

I'm not very good at all this map making but I can try to help if you need a few more people. Contact me.--~Edo Dodo~ 11:34, 8 December 2006 (CST)

Map Zoom Levels

When I play GW in a window, and I make it small, the resolution of the "zoomed in" m-map is lower. Is there a maximum resolution though? This would be good to know for making our base maps. - Lord Ehzed 09:24, 27 November 2006 (CST)

Not sure about window mode but in fullscreen mode you can change the interface size and that will make the map larger. Normal size will be 1:1 and anything larger will just be blown up like the compass. -Smurf 19:27, 27 November 2006 (CST)

Wiki Data vs Database

Ok, well for me to setup the development environment, I need to know if the data will be pulled from a database directly or from a wiki page.

I think it would be much more robust and flexible to pull it directly from a database. Although, a CSV on a Wiki page can accomplish the same thing if necessary. As long as the API knows what goes in which colum such as:

POIName,Lat,Lon,Icon

Then the data could exist directly on the page.

Each "set" of data could exist as a separate page.

The benefit of making it mash data into the GoogleAPI from a Wiki page is that we gain all the history and reversion benefits of the wiki itself. Although, this means that the functionality of the map is tied to the functionality of the Wiki.

Conversely, the Google API could be programmed to read the "text" of the actual wiki article directly from the database instead of parsing the information directly from the Wiki HTML. It would likely be more efficient that way too, since it's a direct database query versus having to load the Wiki HTML, and then parse it.

RFC ErkDog 21:39, 27 November 2006 (CST)

I haven't said much so far, but I've been pondering how to get this working with the wiki to show changes in recentchanges and have a history. Without those, it's not feasible to open up the content to editing by users at large. Having CSVs in a wiki page is definitely easiest but it's kind of crude and will probably end up being a hassle when you want to have a map that's a collection of data from various pages (like a map of all elites as Deep mentioned). At best you end up manually compiling lists (on the bright side, they won't likely change for collectors/skill locations/bosses/quests). But making a database with all the data that we'd like on a map would be a lot of effort. --Fyren 02:13, 28 November 2006 (CST)

I'm kind of still hung up on the bot idea. We'd have all the wiki functionality, and the map could access it's data fast and efficiently (as it's in a database). And while the Wiki is feeding the map with data, they are not dependent on each other - the map could run without the wiki as well, possibly even be used for other games. Of course, it would take some time before the map updates with any wiki changes, which is the main disadvantage I can see. DeepSearch 06:42, 28 November 2006 (CST)

I really think that maintaining individual CSVs of the various "types" of data would be best. I mean, I see Deep's points, but how would the bot know which pages to 'crawl'? And then, that information isn't clearly visible on the page it's a hidden piece of information in the page. It would work to pass coordinates and icon in the URL for maps that appear on specific pages for bosses and such. Actually, you could just pass the items in all the URLs. Plus, there is some kind of limit on the length of the URL I believe. But that seems just as arduous, if not more than having everything of one data type on it's own CSV wiki page. Plus that doesn't accomplish the goal of having a navigable 'world' map in which you can toggle different layers, pan around, zoom in, etc. The worst thing is, the data "source" as it were, has to be sorted out before we can actually proceed with much. PHP could easily be written to simply pass coords within URLS, but then there still needs to be a data source, lol. The ultra -realtimeness- of Deep's approach is probably not of great concern. On individual pages the location of a single entity such as a boss, or multiple entities such as an elite skill's possible locations, could be passed in the URL, and then the bot would crawl it later and add it to the "world" map. ErkDog 08:07, 28 November 2006 (CST)

I haven't read all the detail here because I only just discovered this article exists, but I'm going to propose a solution, and you can let me know if it doesn't make sense ;)

I'm opposed to storing CSVs on the wiki, because I think it's a very painful kind of system.

My suggestion is that for any kind of article that has geographic data we store some co-ordinates (in the same way that we currently store skill articles in a separate template, i.e. Word of Healing and Template:Word of Healing). Then we can produce articles which are JSON representations of the information.

So, for example, we have Josso Essher, which is derived from data in Template:Josso Essher. Then we have something like Monk boss locations (JSON) (or maybe something in a different namespace) which is derived from Template:Josso Essher and other templates.

What do you think? <LordBiro>/<Talk> 10:08, 4 December 2006 (CST)

Then you have a bot crawling pages in order to pop locations into a database. And you also have to include a type of POI and Icon to use in the POI info. So basically, you would have little bits of a CSV scattered across hundreds of pages, instead of centralized and easily modified. Plus you have to code a bot to pop coords into the database. Whereas if it's mashing from the CSV it takes alot of overhead and maintenance out of the system. I still think putting each 'layer' Elites / Mission Maps / Collectors / Etc, on it's own CSV page is the best approach. ErkDog 10:48, 4 December 2006 (CST)

I'm not 100% sure what you mean by popping locations into a database. I guess you are imagining a system where there is a separate database on the same server as a map that's a copy of information from the wiki. That doesn't seem like a very wiki-like solution at all, and I'm opposed to it.

I was suggesting a system whereby, if a user requests a view of all monk boss locations, the map would use XMLHttpRequest to get a JSON/YAML/XML/whatever object from the GuildWiki from an article, i.e. Monk boss locations (JSON). This article would have been produced by deriving information from all monk boss articles, much in the same way we derive monk skills quick reference from every monk skill article. <LordBiro>/<Talk> 14:19, 4 December 2006 (CST)

• sigh*, having a page generated by a bot into the wiki is the same as having the bot pop it in a database, and it's just as damn wiki as the nonsense you are proposing. You are saying putting coords in individual pages and having something coalate them into yet ANOTHER wiki page. Why? It's redundant and pointless. And you can't have the damn map program query dozens of pages to mashup the data. Plus someone has to develop this damn bot to regex ALL the pages on the wiki. Having CSVs is the simplest thing to do. ErkDog 19:19, 4 December 2006 (CST)

You come across as rude, ErkDog, and I don't appreciate it. I will attempt to reply in a civil manner, without sighing or anything.

I was under the impression that the map would be capable of displaying a number of different sets of information. I.e. a list of all monk boss locations, a list of all collectors, etc. If this were the case then I can see two obvious solutions. The first is to write a program in a server-side language that dynamically builds the JavaScript and contains all the code to show the locations of 1 set. So a user might click on "Show Monk boss locations" and they would receive a page containing the JavaScript to display the map, and also all the location data necessary to display the location of Monk bosses. This is not a particularly elegant solution.

The other solution I had envisaged (and the solution which I had perhaps erroneously presumed would be used) is to have a JavaScript map application that would contain no location data, but use XMLHttpRequest (AJAX) to get, for example, all Monk boss locations. So a user would load up a map of Tyria, select to show all Monk boss locations, the map application would make an XMLHttpRequest to some URL, an object containing the Monk boss locations would be returned and interpreted, and displayed to the user. Then the user might also turn on "collector locations". Then they might also turn on "material trader locations". Then they might turn off "collector locations".

The location from which they received this information could be this wiki. By making use of templates we could easily have a list of all Monk bosses in tyria, derived the Monk boss articles, and provided to the JavaScript application in JSON or XML or some other format it could easily interpret.

I should stress that this suggestion would require no bots, no regex and no other server or database, although you would not (as far as I'm aware) be able to carry out searches with this method. <LordBiro>/<Talk> 20:54, 4 December 2006 (CST)

I also want to add that I do not disagree with your statement that using CSVs would be the simplest option, but it is my opinion that the simplest option is not, in this case, the best option. However I will not be the one programming this application, so provided you are polite in your disagreement I'll have no problems. <LordBiro>/<Talk> 20:59, 4 December 2006 (CST)

Perhaps I am unfamiliar with the intricacies of MediaWiki. What you are indicating is by placing templates on 100 different pages, this data can somehow magically be retrieved with a single HTTP query? That doesn't seem very plausible to me, but as I said, I don't know the intricacies of MediaWiki. ErkDog 20:59, 4 December 2006 (CST)

Wow, that was a quick reply! I really should be going to bed now (it's 02:08 in the UK) so I won't reply in detail, but yes it is possible. If you look at the way that Word of Healing and Monk skills quick reference are both derived from Template:Word of Healing then I'm sure you can imagine how we could produce an article that's basically just a list of monk boss templates, and provide them in the form of a JSON object.

As I said, it's not something I'll go into now, but it is possible, and I don't see how it would be any more complicated than the skill system we have in place at the moment. Anyway, nn. <LordBiro>/<Talk> 21:07, 4 December 2006 (CST)

Both options have advantages. Putting the data into the boss pages is where it contextually belongs. It feels like the right thing to do. On the other hand, of course, it comes with a certain overhead that we maybe wouldn't have to deal with when using a central page (in CSV or whatever) to collect all the data. I'm not entirely sure about this either, though. Consider this:
Once we have the boss locations on the map, what happens if the user clicks on one? I would find it nice if there were a little info window popup, showing the basic boss information. Maybe even more, like elite skill and unique item, in multiple tabs (note to self: create a demo showcasing how that could look). Where do we get that information? It seems like even if we were to use a central CSV, we still would need to retrieve data from each of the boss pages to do so. DeepSearch 03:32, 5 December 2006 (CST)

Boss location mockup

I created a quick mockup showing how boss locations could be indicated on the map. This is for the global map of all locations though, and not one that should be placed on the boss article pages. Only tested in Firefox, and the visuals definitely needs some improvement (I just included the Wiki stylesheets for now). Also, you need to disable referrers, otherwise you won't see the images, as the Wiki does not allow deep linking.

http://mapdev.fiftypounds.com/DeepSearch/BossInfoWindowDraft/

To clarify: There is a small Assassin icon in Crystal Overlook ;) DeepSearch

Glad to see some progress being made :) How is that data being mashed into the map API? The popup when you click the icon is beyond hard core. Although, how do we circumvent this behavior where images don't show up? Is this a configuration setting that Fyren can change? What you appear to be considering "deep linking" is actually hot linking. The wiki sees a non-self referral and goes no way jose. Fyren, can you add an allowance for hot linking from mapdev.fiftypounds.com? Disabling the referrer header just tricks the Wiki into believing that the request for the image is a "Direct Request" versus from an off site page. I will send Fyren a note to check this section about adding mapdev.fiftypounds.com as an allowed "referrer". ErkDog 12:07, 7 December 2006 (CST)

Done. --Fyren 12:17, 7 December 2006 (CST)

Thanks Fyren :) ErkDog 22:09, 7 December 2006 (CST)

The data is pretty much hardcoded (it's loaded via ajax from text files, but the current code would only support one boss icon). It is meant more as an example how it could possibly look visually, not so much as a codebase that would actually be used later. And yes, I meant hotlinking. Sorry about that ;) DeepSearch 12:41, 7 December 2006 (CST)

Also, by deciding what features we need, and what data we want to show, I hope it will also help as determine which data the map needs to access and what might be the best way to do so. DeepSearch 12:46, 7 December 2006 (CST)

License

As I'm planning on doing some actual coding in the near future - under what license do we want to put all this? CC by-nc-sa like the Wiki itself? Personally, I think I would prefer a by-sa-License that allows commercial use as well, or maybe more traditional, the GPL. DeepSearch 11:59, 7 December 2006 (CST)

The code itself is not accessible by anyone viewing the map. And it becomes a moot point if we don't publish all the code and scripts for public consumption. But, code falls under a bit of a different realm than information collected and assimilated from a game. I would think being code, like most other code, a GNU/GPL licensing would be most appropriate. Commercial entities can use code released under the GNU/GPL, and they can even profit from it's use. Just not the direct sale of the code itself. For example, Google uses almost exclusively opensource GNU/GPL offerings for their servers, with the exception of what they code in house. They profit greatly from the use of OS/GNU/GPL, but don't directly sell any of the code they are using. ErkDog 12:14, 7 December 2006 (CST)

"The code itself is not accessible by anyone viewing the map." - I'm not sure what you mean by that. The Javascript code would of course be accessible. Also, with possibly multiple contributors, I'd assume some kind of arrangement would have to be made. DeepSearch 12:58, 7 December 2006 (CST)

Map tiles uploaded

I finally got all the tile created and upload and here are the results:

Map	Width	Height	Size
Tyria	8178	6144	92.6 MB	view
Cantha	6144	4608	52.7 MB	view
Elona	6152	4611	56.6 MB	view
The Battle Isles	2048	1536	3.9 MB	view
Pre-Searing Ascalon	2048	1538	7.6 MB	view

What do you guys think? -Smurf 15:46, 9 December 2006 (CST)

slick as heck dood :) ErkDog 16:54, 9 December 2006 (CST)