Difference between revisions of "Wiimms mkw-ana (tool)"

From Custom Mario Kart
Jump to navigation Jump to search
(mkw-ana v0.08)
Line 16: Line 16:
 
|-
 
|-
 
! Current Version:
 
! Current Version:
−
| 0.07, 2013-02-24
+
| 0.08, 2013-03-14
 
|}
 
|}
  
Line 120: Line 120:
 
   if a BMG with names is loaded.
 
   if a BMG with names is loaded.
  
 +
 +
mkw-ana v0.08 r1509 - 2013-03-14
 +
 +
- Detection of different network packet types:
 +
  ARP, IP, IMCP, TCP, UDP and DNS(UDP).
 +
- The engine class (50cc, 100cc, 150cc, mirror) is now detected.
 +
- The 2 new options --receive-ip and --send-ip allow to specfy addresses as
 +
  output filter like --receive and --send for the home client.
 +
- New command: DUMP0: Print a raw dump of all packets. This dump can be used
 +
  for all network dumps, not only for MKWii.
 +
- New command: DNS: Print DNS and optional ARP packets in human readable
 +
  format. This command can be used for all network dumps, not only for MKWii.
 +
- Command 'HELP COMMANDS' prints a list of all commands and 'HELP OPTIONS'
 +
  a list of all global and command specific options.
  
 
</pre></spoiler>
 
</pre></spoiler>
Line 129: Line 143:
 
<spoiler><pre>
 
<spoiler><pre>
  
−
mkw-ana v0.07/x86_64 r1475 -- Dirk Clemens -- 2013-02-24
+
mkw-ana v0.08/x86_64 r1509 -- Dirk Clemens -- 2013-03-14
 
--------------------------------------------------------
 
--------------------------------------------------------
  
Line 135: Line 149:
  
 
Syntax: mkw-ana [option]... command [option|parameter|file]...
 
Syntax: mkw-ana [option]... command [option|parameter|file]...
 +
  
 
Commands:
 
Commands:
  
−
   VERSION       : Print program name and version and exit.
+
   VERSION         : Print program name and version and exit.
−
   HELP      | H : Print help and exit. If the first non option is a valid
+
   HELP      | H   : Print help and exit. If the first non option is a valid
−
                  command name, then a help for the given command is printed.
+
                    command name, then a command specific help is printed.
−
                  Command 'HELP OPTIONS' prints a list with all options.
+
                    Command 'HELP COMMANDS' prints a list of all commands and
−
   ARGTEST       : This debug command accepts all kinds of parameters and
+
                    'HELP OPTIONS' a list of all global and command specific
−
                  prints one line for each parameter.
+
                    options.
−
   TEST           : Test options: All options are allowed, some are printed.
+
   ARGTEST         : This debug command accepts all kinds of parameters and
 +
                    prints one line for each parameter or option.
 +
   TEST           : Test options: All options are allowed, some are printed.
 +
  ERROR    | ERR : Translate exit codes to message names. If no exit code is
 +
                    entered, print a table with all error messages.
  
−
   DUMP1    | D1 : Packet based hex dumper.
+
  DUMP0    | D0  : Print a raw dump of all packets. This dump can be used for
−
   DUMP2    | D2 : First record base hex dumper.
+
                    all network dumps, not only for MKWii.
−
   DUMP3    | D3 : New and improved variant of DUMP2 (record based).
+
   DUMP1    | D1 : Packet based hex dumper.
−
   DUMP      | D : Use the best/latest dumping mode (depends on options). At
+
   DUMP2    | D2 : First record based hex dumper.
−
                  the moment DUMP is an alias for DUMP3.
+
   DUMP3    | D3 : New and improved variant of DUMP2 (record based).
 +
   DUMP      | D   : Use the best/latest dumping mode (depends on options). At
 +
                    the moment DUMP is an alias for DUMP3.
  
−
   FLOWRATES | F : Print flowrates for all network dumps, not only for MKWii.
+
   FLOWRATES | F   : Print flowrates of the data traffic. This command can be
 +
                    used for all network dumps, not only for MKWii.
 +
  DNS            : Print DNS and optional ARP packets in human readable
 +
                    format. This command can be used for all network dumps, not
 +
                    only for MKWii.
  
−
   CLIENTS  | C : List all clients.
+
   CLIENTS  | C   : List all clients.
−
   USERS    | U : List all users.
+
   USERS    | U   : List all users.
−
   RECORDS  | R : Print all record names.
+
   RECORDS  | R   : Print all record names.
−
   STAGES    | S : Print all stages.
+
   STAGES    | S   : Print all stages.
−
   LOG      | L : Print all stages including tables.
+
   LOG      | L   : Print all stages including tables.
−
   TRACKS         : Print all track selections.
+
   TRACKS         : Print all track selections.
−
   TOTALS    | T : Print all totals as text dump.
+
   TOTALS    | T   : Print all totals as text dump.
  
−
   RACE           : Live dump of senders race statitics to the screen.
+
   RACE           : Live hex dump of senders race statistics to the screen.
  
 
Type 'mkw-ana HELP command' to get command specific help.
 
Type 'mkw-ana HELP command' to get command specific help.
Line 170: Line 195:
 
   -V --version      Stop parsing the command line, print a version info and
 
   -V --version      Stop parsing the command line, print a version info and
 
                     exit.
 
                     exit.
−
   -h --help        Stop parsing the command line, print a help message and
+
   -h --help        Print help and exit. If the first non option is a valid
−
                    exit.
+
                    command name, then a help for the given command is printed.
 
     --xhelp        Stop parsing the command line and print a help message with
 
     --xhelp        Stop parsing the command line and print a help message with
 
                     all commands included. Exit after printing.
 
                     all commands included. Exit after printing.
Line 184: Line 209:
 
                     The impact is command dependent.
 
                     The impact is command dependent.
 
   -A --allow-all    Usually commands accept only options with impact to the
 
   -A --allow-all    Usually commands accept only options with impact to the
−
                     command. All other options firce an syntax error. But if
+
                     command. All other options fire an syntax error. But if
 
                     --allow-all is set, all commands accept all options.
 
                     --allow-all is set, all commands accept all options.
 
     --old          Use old implementation if available. All previous --new are
 
     --old          Use old implementation if available. All previous --new are
Line 208: Line 233:
 
     --combine      Logical combine network dumps to one single dump before
 
     --combine      Logical combine network dumps to one single dump before
 
                     executing options --skip and --term.
 
                     executing options --skip and --term.
−
    --follow        Don't close the last input dump on reaching end of file.
+
  -f --follow        Don't close the last input dump on reaching end of file.
 
                     Instead wait for appended data. This works like the unix
 
                     Instead wait for appended data. This works like the unix
 
                     tool 'tail -f'.
 
                     tool 'tail -f'.
−
     --ip addr      Define an IP (or DNS name) for filtering. Only packets
+
     --ip addr      Define an address (IP or DNS name) for filtering. Only
−
                     from an to this host are accepted, all others are ignored.
+
                     packets from an to this host are accepted, all others are
−
     --home addr    Define an IP (or DNS name) as home client.
+
                    ignored.
 +
     --home addr    Define an address (IP or DNS name) as home client.
 
                       Without this options, the tool tries to determine the
 
                       Without this options, the tool tries to determine the
 
                     home client by analysing sender and receiver of the first
 
                     home client by analysing sender and receiver of the first
Line 219: Line 245:
 
                     192.168/16, 169.254/16) has priority over a non local
 
                     192.168/16, 169.254/16) has priority over a non local
 
                     network. If sender and receiver have the same priority,
 
                     network. If sender and receiver have the same priority,
−
                     the sender is used.
+
                     the IP of the sender is used.
−
     --wii addr      Define an IP (or DNS name) ad home client and for
+
     --wii addr      Define an address (IP or DNS name) as home client and for
−
                     filtering. This options is a shortcut for '--ip addr
+
                     filtering. This options is a shortcut for '--home addr
−
                     --home addr'.
+
                     --ip addr'.
−
     --write file    Write filtered network packets as PCAP to the 'file'.
+
     --write file    Write filtered network packets as PCAP v2.4 to 'file'.
  
−
   -p --no-proxy      Ignore proxy packets.
+
   -p --no-proxy      Don't dump proxy packets (packets, which contains a PROXY
−
   -r --receive      Select only packtes received by the home client. If non or
+
                    record).
−
                     both of --receive and --send are set, this filter is
+
   -r --receive      Dump only network packets received by the home client
−
                     deactivated.
+
                     (option --home) or allowed by --send, --receive-ip or
−
   -s --send          Select only packtes send by the home client. If non or
+
                     --send-ip.
−
                     both of --receive and --send are set, this filter is
+
   -s --send          Dump only network packets send by the home client (option
−
                     deactivated.
+
                    --home) or allowed by --receive, --send-ip or
−
   -L --length ranges Select UDP packets by UDP data length. The 8 bytes long
+
                    --receive-ip.
−
                    UDP header does not count.
+
    --receive-ip addr
 +
                    Dump only network packets received by the entered address
 +
                    (IP or DNS name) or allowed by --send-ip, --receive or
 +
                     --send. --rip is a short cut for --receive-ip.
 +
    --send-ip addr  Dump only network packets send by the entered address (IP
 +
                    or DNS name) or allowed by --receive-ip, --send or
 +
                     --receive. --sip is a short cut for --send-ip.
 +
   -L --length ranges Dump only UDP packets with specified UDP data length. The
 +
                    8 bytes long UDP header does not count.
 
                       The parameter is a comma separated list of INDEX,
 
                       The parameter is a comma separated list of INDEX,
 
                     INDEX1:, INDEX1:INDEX2 and INDEX#LENGTH elements.
 
                     INDEX1:, INDEX1:INDEX2 and INDEX#LENGTH elements.
−
   -S --stage list    Accept UDP packets only, if one of the entered stages is
+
   -S --stage list    Dump UDP packets only, if one of the entered stages is
 
                     active.
 
                     active.
−
                       The parameter is a comma separated list of stage names.
+
                       The parameter is a comma separated list of stage names,
−
                     A stage list is not available yet; use the dumps to
+
                    optional preceeded by '+' (enable) or '.' (disable). Type
 +
                     'mkw-ana test' for a list of stages or use the dumps to
 
                     identify stage names.
 
                     identify stage names.
−
   -t --type list    Select UDP packets by record types. The complete packet is
+
   -t --type list    Dump UDP packets only, if at least one record of the
−
                    selected if at least one of the entered match.
+
                    packet match the entered record list.
−
                       The parameter is a comma separated list of record names.
+
                       The parameter is a comma separated list of record names,
−
                     A record list is not available yet; use the dumps to
+
                    optional preceeded by '+' (enable) or '.' (disable). Type
 +
                     'mkw-ana test' for a list of records or use the dumps to
 
                     identify record names.
 
                     identify record names.
 
   -T --TYPE list    Same as --type except for command DUMP3.
 
   -T --TYPE list    Same as --type except for command DUMP3.
Line 259: Line 295:
 
                     Dependent of option --long the foramts are: 'SSSSS',
 
                     Dependent of option --long the foramts are: 'SSSSS',
 
                     'SSSSS.s' or 'SSSSS.sss'
 
                     'SSSSS.s' or 'SSSSS.sss'
 +
    --rel-info      Like --rel, but reset the origin whenever a reference time
 +
                    is defined in the info file.
 
   -1 --one-line      Print the hexdumps as one line for each object. This makes
 
   -1 --one-line      Print the hexdumps as one line for each object. This makes
−
                     the dumps horizontal very large, but is good for comparing
+
                     the dumps horizontal very large, but it is good for
−
                     objects of the same type. Very helpful is to pipe the
+
                     comparing objects of the same type. Very helpful is to
−
                    output to 'less -S', which supports horizontal scrolling.
+
                    pipe the output to 'less -S', which supports horizontal
 +
                    scrolling.
 
                       If set twice, some record types are not split into sub
 
                       If set twice, some record types are not split into sub
 
                     records.
 
                     records.
−
   -2 --sep-lines    Dump one line per record (like option --one-line) an empty
+
   -2 --sep-lines    Dump one line per record (like option --one-line) and an
−
                     line between packtes.
+
                     empty line between packets.
 
   -a --ascii        Append an ASCII character dump behind the hexdump. This
 
   -a --ascii        Append an ASCII character dump behind the hexdump. This
 
                     option is ignored if using --one-line.
 
                     option is ignored if using --one-line.
−
   -x --hex-only      Some known values a printed in native format instead as
+
   -x --hex-only      Some known values, records or packets are printed in
−
                    hex number. If option --hex-only is set, all data is
+
                    native format instead as hex number. If option --hex-only
−
                    printed as hex number.
+
                    is set, all data is printed as hex number. --hex is a
−
   -d --delta        If set, record data is comapred with the data of the
+
                    short cut.
 +
   -d --delta        If set, record data is compared with the data of the
 
                     previous record of same type and client. If a byte is
 
                     previous record of same type and client. If a byte is
−
                     unchanged, a '--' is printed inted of 2 hex digits.
+
                     unchanged, a '--' is printed intead of 2 hex digits.
 
   -I --index ranges  Dump only bytes selected by the range list. This make the
 
   -I --index ranges  Dump only bytes selected by the range list. This make the
 
                     hex dump smaller especially for one-line dumps.
 
                     hex dump smaller especially for one-line dumps.
Line 294: Line 334:
 
                     create different files.
 
                     create different files.
 
     --sleep float  Sleep 'float' seconds after a race has finished. This
 
     --sleep float  Sleep 'float' seconds after a race has finished. This
−
                     options slow down a simulation direct after logging and
+
                     option slows down a simulation run direct after logging
−
                     printing the race statistics.
+
                     and printing the race statistics.
  
−
     --mii dir      Extract Miis to the existing directory 'dir'. Existing Mii
+
     --mii dir      Extract Miis to the already existing directory 'dir'.
−
                    files will be overwritten.
+
                    Existing Mii files will be overwritten.
 
</pre></spoiler>
 
</pre></spoiler>
  
Line 330: Line 370:
 
=== Options ===
 
=== Options ===
 
=== Download ===
 
=== Download ===
−
Here is the v0.07 of the tool mkw-ana:
+
Here is the v0.08 of the tool mkw-ana:
−
[http://download.wiimm.de/mkw-ana/mkw-ana-v0.07-r1475.zip mkw-ana-v0.07-r1475.zip] (305 kB)
+
[http://download.wiimm.de/mkw-ana/mkw-ana-v0.08-r1509.zip mkw-ana-v0.08-r1509.zip] (320 kB)
 
* It contains binaries for:
 
* It contains binaries for:
 
** Linux i386
 
** Linux i386

Revision as of 16:30, 14 March 2013

Wiimms mkw-ana
File:Wiimms-SZS-Tools.png
Author: Wiimm
Operating Systems: Linux (i386,x86_64),
Windows (cygwin).
Software Type: Mario Kart Wii network analyzer
File Formats: PCAP 2.4 (tcpdump), BMG (text).
Current Version: 0.08, 2013-03-14

The tools mkw-ana is a new project by Wiimm to analyze the network protocol of Mario Kart Wii.



In Progress
This article is managed by Wiimm, the developer, and will grow slowly. Feel free to edit typos and grammatical or speech issues.

Intention

In November 2012, Wiimm decided to analyze the network protocol of Mario Kart Wii. The main goal was to discover online cheaters. Another goal is to set up his own server if any time Nintendo will shut down its servers.

So the main feature is to dump the packets of a tcpdump (done by tcpdump or wireshark) in a user-friendly format. After first experiments, it becomes also a live racing statistic tool.

Terms and definition

Template:MKWii Network Terms

The Tool

Change log

Show text


Built-in Help

Let's start with the built-in help as an overview about the tool:

Show text


General Description

The tool started as simple hex dumper reading network dumps in PCAP format. In the first phase of the tool, the textual dumps of whireshark and tcpdump were much better. But after only a few days, the tool learned to handle records, clients, users, friend codes and Miis. From this moment the tool was better to analyze the Mario Kart Wii traffic.

Now, mkw-ana split the traffic into records and scans some data to detect stages of the online meeting. Stages are for example room, prepare race, count down, racing and end of race. It is able to separate races into events (grand prix and team rand prix) and to calculate racing tables. Racing data can also be exported to support live statistics.

At the moment there are three different kinds of hexdumps. All 3 are able to dump in one line mode to have large tables. Tool less is here a very good tool for vertical and horizontal scrolling. The stages are includes into the dump as comment lines. The dumped records can be filters by sending, receiving, proxy, record types, stage types and packet length. It is also possible to select the dumped bytes by indices and ranges.

Another feature is, that mkw-ana can read comment files. If making videos of the dumped meetings, you can write such comment file. Each line starts with a timestamp followed by a comment. Virtual Dub is a good tool for this job. Then you must synchronize the comment file with the network dump. The start of the first game ("GO" in the video) is a very good point for synchronization. Here is an example of a comment file (in german): 

>2012-12-05 19:10:39.745 - 12:22.792
0:00:00.000 Video Start

0:12:22.792 Rennen 1.1, GO!
0:12:54.123 Tinti wird angekündigt, T=0:31.322
0:14:17.924 Power wird angekündigt, T=1:55.120
0:15:33.633 Blitz schlägt ein, T=1:35.326
...
Notes
  • The first line ist the real time of the start of the race minus the video time stamp. This is the synchronisation. An synchronisation can be done multiple times.
  • The line with video timestamp and comment follow. The focus of the comment change as the point of interest.

???

Dumping Data

Options

Download

Here is the v0.08 of the tool mkw-ana: mkw-ana-v0.08-r1509.zip (320 kB)

  • It contains binaries for:
    • Linux i386
    • Linux x86_64
    • Cygwin/Windows (A Cygwin system must be installed).
  • Some scripts as examples.
  • Some BMG text examples.
  • Some doc files.

It is an alpha distribution and and only minimal support will be given at the moment.

Capture the network data

First you must capture the network traffic of the Wii. Therefore you must redirect it to a PC running a capture software. There are 3 general ways to to this:

  • If you have a manageable switch, enable port mirroring and send all Wii traffic to a PC.
  • Use your PC as router.
  • Use old network hubs (not switches). A hub will mirror all traffic of all ports to all others; it's just a multi port repeater and will slow down your network.

Use a software like tcpdump or wireshark to capture the data. Best is to save the captured data directly to a file or to send it to other commands like mkw-ana for a live analysis.

It's also possible to save the data to a file and to make a live analysis at the same time. Use the following command pipe: 

tcpdump -w- -U -i eth1 host wii | tee save.dump | mkw-ana ...

If using wireshark, save the dump to a file and use the following command for a live analysis: 

mkw-ana --follow DUMPFILE ...

Live Statistics

mkw-ana can scan the network traffic in real time and can produce makedoc or php data files. Together with ssh and an cgi script, a live statistic is created. Live means that the tables are updates 2-5 seconds after the race have finished.

How it works

The whole job is done by 3 processes:

  1. First, you must capture the network traffic like described above. Then use one of the commands: 
    ... | mkw-ana log --md  DATAFILE
... | mkw-ana log --php DATAFILE
mkw-ana --follow DUMPFILE log --md  DATAFILE
mkw-ana --follow DUMPFILE log --php DATAFILE
  2. Each time, a new DATAFILE is written, it must be transferred to the web server. A script using shh, sftp scp or ftp within an endless loop will do this job automatically.
  3. Last not least, a CGI or PHP script running at the web server must read the data files to serve a html-page to the visitors.

To see, what live means, visit the live statistics on Wednesday or Thursday between 19:10 and 20:30 CET (Central European Time).

Links

Retrieved from "https://wiki.tockdom.com/w/index.php?title=Wiimms_mkw-ana_(tool)&oldid=33724"