Difference between revisions of "Wiimms mkw-ana (tool)"

From Custom Mario Kart
Jump to navigation Jump to search
m
Line 33: Line 33:
 
== Intention ==
 
== Intention ==
  
−
In November 2012, [[Wiimm]] decided to analyze the network protocol of [[Mario Kart Wii]]. The main goal was to discover online cheaters. Another goal was to set up his own server if [[Nintendo]] ever shuts down its servers; and on May 20th, 2014 Nintendo done so.
+
In November 2012, [[Wiimm]] decided to analyze the network protocol of [[Mario Kart Wii]]. The main goal was to discover online cheaters. Another goal was to set up his own server if [[Nintendo]] ever shut down its servers, which eventually happened on May 20th, 2014.
  
−
So the main feature is to dump the packets of a tcpdump (done by ''tcpdump'' or ''wireshark'') in a user-friendly format. After first experiments, it also became a ''live racing statistic tool''.
+
The main feature is to dump the packets of a tcpdump (done by ''tcpdump'' or ''wireshark'') in a user-friendly format. After first experiments, it also became a ''live racing statistic tool''.
  
 
=== Terms and definition ===
 
=== Terms and definition ===

Revision as of 15:59, 24 June 2015

Wiimms mkw-ana
File:Wiimms-SZS-Tools.png
Author: Wiimm
Operating Systems: Linux (i386,x86_64),
Windows (Cygwin).
Software Type: Mario Kart Wii network traffic analyzer
File Formats: PCAP 2.4, PCAP 2.4.modified, PCAP-NG, bzip2 compression, BMG (text).
Current Version: v1.02, 2014-12-27
Current Beta: v1.03-beta3, 2015-06-19
Homepage: mkw-ana.wiimm.de

The tools mkw-ana is a new project by Wiimm to analyze the network protocol of Mario Kart Wii.



Intention

In November 2012, Wiimm decided to analyze the network protocol of Mario Kart Wii. The main goal was to discover online cheaters. Another goal was to set up his own server if Nintendo ever shut down its servers, which eventually happened on May 20th, 2014.

The main feature is to dump the packets of a tcpdump (done by tcpdump or wireshark) in a user-friendly format. After first experiments, it also became a live racing statistic tool.

Terms and definition

Template:MKWii Network Terms

The Tool

Change log

mkw-ana v1.03.beta3 r2128 - 2015-06-19

 - Over the past few months I have collected new knowledge about status records.
   And with this new knowledge I have re-implemented the client and race slot
   detection. Now global/worldwide are correct interpreted.
   Overall, this is in progress. So this version is EXPERIMENTAL.

 - Option --print=modes: 2 more debugging modes added:
    - SLOT: Print client and race slot as first column. 
    - PID:  Print a column with the profile id.

 - Option --log-mode=modes: 3 more modes added:
    - SLOTS:  Print logging lines for slot relevant changes.
    - USLOTS: Print a refreshing table with used client and racing slots.
              Try: mkw-ana log --lmd USLOTS <DUMPFILE>
    - RSLOTS: Like as USLOTS, but without refreshing. Command RACE handles
              USLOTS and RSLOTS in the same, but special way.

 - Command FC supports now NDS checksums. The checksum algorithm is selected
   by an internal game list. It can be overridden by new option --cmode=mode.
   Overall, the kind of working and list generation has changed.

 - The dump commands try to detect the current game to use the correct friend
   code calculation.

 - New command PID: Print a pid for each argument.

 - Support of external files for game ID4 management:
    - New options: --gimport=file: Load the game list from a text file.
    - New options: --gexport=file: Save the game list to a text file.
    - Autoload of files /usr/local/share/mkw-ana/gameid.txt (global setup)
      and ~/.mkw-ana/gameid.txt (private setup).

 - Command QUERY supports now imported and exported secret lists.
    - New options: --simport=file: Load the secret list from a text file.
    - New options: --sexport=file: Save the secret list to a text file.
    - Autoload of files /usr/local/share/mkw-ana/secret.txt (global setup)
      and ~/.mkw-ana/secret.txt (private setup).

 - New command UPDATE: Updates the Game-ID and the SECRET lists. It works only
   if tool 'wget' is available.

 - New color management.

 - Option --dwc is not longer supported (network address changes frequently).
   Use --server=ip or --server=domain instead.


mkw-ana v1.02 r2015 - 2014-12-27

 - Status records of types (hex) 01, 02, 03, 07, 82 and 83 are dumped with
   member names: http://wiimm.de/r/ctw/MKWii_Network_Protocol/STATUS

 - Option --new: If set, a new slot assigning algorithm based on the status
   messages 02, 07, 82 and 83 is activated. This new algorithm will improve
   the analysis of global races. In this version the new algorithm prints only
   log messages for comparison and has no other impact.

→ old logs


General Description

The tool started as simple hex dumper reading network dumps in PCAP format. In the first phase of the tool, the textual dumps of wireshark and tcpdump were much better. But after only a few days, the tool learned to handle records, clients, users, friend codes and Miis. From this moment the tool was better to analyze the Mario Kart Wii traffic.

Now, mkw-ana split the traffic into records and scans some data to detect stages of the online meeting. Stages are for example room, prepare race, count down, racing and end of race. It is able to separate races into events (grand prix and team grand prix) and to calculate racing tables. Racing data can also be exported to support live statistics.

At the moment there are three different kinds of hexdumps. All 3 are able to dump in one line mode to have large tables. Tool less is here a very good tool for vertical and horizontal scrolling. The stages are includes into the dump as comment lines. The dumped records can be filters by sending, receiving, proxy, record types, stage types and packet length. It is also possible to select the dumped bytes by indices and ranges.

Include logs into output

Another feature is, that mkw-ana can read comment files. If making videos of the dumped meetings, you can write such comment file. Each line starts with a timestamp followed by a comment. VirtualDub[1] is a good tool for this job. Then you must synchronize the comment file with the network dump. The start of the first game ("GO" in the video) is a very good point for synchronization. Here is an example of a comment file (in german): 

# All lines beginning with an '#' are comments and ignored.

# Sync:       timestamp of dump - timestamp of video
> 2014-03-07 18:33:31.583 +0100 - 0:08:09.667

0:00:00.000  Video Start
0:00:54.600  Enter WFC first time
...
0:08:09.667  GO!   GP 1.1

Notes:

  • The first line is the real time of the start of the race minus the video time stamp. This is the synchronisation. An synchronisation can be done multiple times.
  • The lines with the video timestamp and comment follow.
  • The name of the comment file must be the same as the network dump, but it must have the extension ".info" instead of ".eth".
  • Here you can find example dumps and log files: http://download.wiimm.de/mkw-ana/dumps/

Download

You can find the latest and some old distributions here
Content
  • Binaries for:
    • Linux i386
    • Linux x86_64
    • Cygwin/Windows (Needed Cygwin[2] DLL files are delivered. Best is to install a Cygwin system).
  • Some scripts as examples.
  • Some BMG text examples.
  • Some doc files.
Sometimes I upload single tool updates (beta versions) for testers

Capture the network data

First you must capture the network traffic of the Wii. Therefore you must redirect it to a PC running a capture software. There are 3 general ways to to this:

  • If you have a manageable switch, enable port mirroring and send all Wii traffic to a PC.
  • Use your PC as router.
  • Use old network hubs (not switches). A hub will mirror all traffic of all ports to all others; it's just a multi port repeater and will slow down your network.

Use a software like tcpdump or wireshark to capture the data. Best is to save the captured data directly to a file or to send it to other commands like mkw-ana for a live analysis.

It's also possible to save the data to a file and to make a live analysis at the same time. Use the following command pipe: 

tcpdump -w- -U -i eth1 host wii | tee save.dump | mkw-ana ...

It is important to filter the data by host ip_or_name, because foreign traffic interfere the wii traffic analysis and will have curious side effects.

If using wireshark, save the dump to a file and use the following command for a live analysis: 

mkw-ana COMMAND --follow DUMPFILE ...

See »Dumping Network Traffic« more more details.

Accepted file formats

mkw-ana accepts the following file formats for the network dumps:

  • PCAP 2.4 : Standard packet capturing file format[3].
    • Big and little endian are supported.
    • Timestamps in micro- and in nanoseconds are supported.
  • PCAP 2.4.modified : Like PCAP, but with an extend packet header. This format is used by several routers, AVM FRITZ!Box[4] is one example.
    • Big and little endian are supported.
    • Only microseconds timestamps are supported.
  • PCAP-NG 2.4.modified : A next generation (NG) PCAP format[5].
    • Big and little endian are supported.
    • Microseconds timestamps are assumed, other are not supported.
    • Only the Enhanced Packet Block[6] is supported to retrieve packets.
  • BZIP2 compression
    • mkw-ana detects a BZIP2 compression automatically. It is supported for all other dump file formats.

mkw-ana accpets any list of dump files. The file format is detected for each single input file, so mixed formats are possible. The special file name »-« (minus sign) means: Don't open the file and and read the standard input (stdin) instead. So one of the input files can be read via pipe.

Live Statistics

mkw-ana can scan the network traffic in real time and can produce makedoc or php data files. Together with ssh and an cgi script, a live statistic is created. Live means that the tables are updates 2-5 seconds after the race have finished.

How it works

The whole job is done by 3 processes:

  1. First, you must capture the network traffic like described above. Then use one of the commands: 
    ... | mkw-ana log --md  DATAFILE
... | mkw-ana log --php DATAFILE
mkw-ana --follow DUMPFILE log --md  DATAFILE
mkw-ana --follow DUMPFILE log --php DATAFILE
  2. Each time, a new DATAFILE is written, it must be transferred to the web server. A script using ssh, sftp scp or ftp within an endless loop will do this job automatically.
  3. Last not least, a CGI or PHP script running at the web server must read the data files to serve a html-page to the visitors.

To see, what live means, visit the live statistics on Wednesday or Thursday between 19:10 and 20:30 CET (Central European Time).

Links

References

  1. ↑ Wikipedia: VirtualDub, a free video utility.
  2. ↑ Cygwin, a Linux like environment for Windows.
  3. ↑ Wireshark: LIBPCAP file format
  4. ↑ AVM FRITZ!Box, a widely used router in Germany
  5. ↑ Wincap: The next generation PCAP file format
  6. ↑ Wincap: PCAP-NG, Enhanced Packet Block


Template:MKWii Network Protocol

Retrieved from "https://wiki.tockdom.com/w/index.php?title=Wiimms_mkw-ana_(tool)&oldid=87054"