Difference between revisions of "Wiimms mkw-ana (tool)"

From Custom Mario Kart
Jump to navigation Jump to search
m
(mkw-ana v0.12)
Line 7: Line 7:
 
|-
 
|-
 
! Operating Systems:
 
! Operating Systems:
−
| Linux (i386,x86_64),<br/>Windows (cygwin).
+
| Linux (i386,x86_64),<br/>Windows (Cygwin).
 
|-
 
|-
 
! Software Type:
 
! Software Type:
Line 13: Line 13:
 
|-
 
|-
 
! File Formats:
 
! File Formats:
−
| PCAP 2.4 (tcpdump), [[BMG]] (text).
+
| PCAP 2.4 (tcpdump), bzip2 compression, [[BMG]] (text).
 
|-
 
|-
 
! Current Version:
 
! Current Version:
−
| 0.11, 2013-07-14
+
| 0.12, 2013-08-19
 
|}
 
|}
  
Line 42: Line 42:
  
 
<pre>
 
<pre>
−
mkw-ana v0.11 r1622 - 2013-07-14
+
mkw-ana v0.12 r1644 - 2013-08-19
−
 
 
−
- More record types are detected now, some only as unnamed hex-records.
 
−
 
 
−
- The 'PREFIX*' records are renamed to 'ROOM*', because they are only send in
 
−
  room events. For non room events, there exists the prefix record 'GLOBAL'.
 
−
 
 
−
- Optimized event detection, including world wide races and battles.
 
−
 
 
−
- A first battle stage detection works now. Battle records are different from
 
−
  racing rcords and not analysed yet.
 
−
 
 
−
- New command: COUNT: Count the record types. Togther with option --verbose
 
−
  it shows all defined record types with short and long names.
 
−
 
 
−
- Event and race numbers are now incremented at begin of an event.
 
−
 
 
−
- Track, driver, vehicle and engine names are now available in (us)english
 
−
  and german without loading a BMG text file. The default output language is
 
−
  english. If option --de is set, the default output language is switched to
 
−
  german. However, a loaded BMG text file may override all names.
 
−
 
 
−
- Bug fix: Daylight saving is now recognized.
 
  
−
  - The checksum of each UDP packet is now checked and invalid packets are
+
  - Bug fix: For aligment, the Mii data in user records is 0x4c bytes long,
−
   dropped silently. If option --checksum is set, the packets are not dropped.
+
   2 bytes more than the Mii file size (0x4a=74).
−
  This makes only sense for hex dumps or playing with Dolphin. If option
+
- Detection of new records: ITEM_CTRL, ITEM_LIST, ITEM_DATA.
−
   --checksum is set twice, the checksums are not calculated and assumed to be
+
- First item cheat detection, only visible for command RACE. It produce
−
  correct.
+
   false positives, but if the counter is >5, a cheater play with you.
−
 
+
  - Command RACE: New output with head line, speed and item cheat infos.
−
  - Some minor printing and counting bugs fixed.
+
  - New option for command RACE: --print=list (-P list):
−
 
+
    Print only the specified columns of the output table. A comma separated
−
  - Windows only: Cygwin update to v1.7.20 2013-06-07.
+
    list of keywords is expected: RANK, TIME, POS=X-POS+Y-POS+Z-POS, DIR,
−
 
+
    SPEED=3D-SPEED,H-SPEED, STATUS, DRIFT=D-COUNT+D-CHEAT,
−
- Windows only: The needed Cygwin DLL files are now delivered too.
+
    XDRIFT=DRIFT+D-MINTIME, ITEM=I-CHEAT, CHEAT=D-CHEAT,I-CHEAT TEST, FC,
 +
    MINI-FC and NAME. Also available: CLEAR, MIN, DEFAULT, MAX and ALL.
 
</pre>
 
</pre>
  
 
;Old change log:
 
;Old change log:
 
<spoiler><pre>
 
<spoiler><pre>
−
mkw-ana v0.08 r1509 - 2013-03-14
 
−
- Detection of different network packet types:
 
−
  ARP, IP, IMCP, TCP, UDP and DNS(UDP).
 
−
- The engine class (50cc, 100cc, 150cc, mirror) is now detected.
 
−
- The 2 new options --receive-ip and --send-ip allow to specify addresses as
 
−
  output filter like --receive and --send for the home client.
 
−
- New command: DUMP0: Print a raw dump of all packets. This dump can be used
 
−
  for all network dumps, not only for MKWii.
 
−
- New command: DNS: Print DNS and optional ARP packets in human readable
 
−
  format. This command can be used for all network dumps, not only for MKWii.
 
−
- Command 'HELP COMMANDS' prints a list of all commands and 'HELP OPTIONS'
 
−
  a list of all global and command specific options.
 
−
 
−
 
 
mkw-ana v0.09 r1532 - 2013-05-04
 
mkw-ana v0.09 r1532 - 2013-05-04
  
Line 110: Line 75:
 
   presentaion as part of the hex dumps.
 
   presentaion as part of the hex dumps.
 
  - Some minor bugs fixed.
 
  - Some minor bugs fixed.
 +
  
 
mkw-ana v0.10 r1577 - 2013-06-01
 
mkw-ana v0.10 r1577 - 2013-06-01
 +
 
  - Complete new RACE command: Print a live race table of all players with the
 
  - Complete new RACE command: Print a live race table of all players with the
 
   following infos:
 
   following infos:
Line 126: Line 93:
 
  - Option --delta works now nibble based instead of byte based.
 
  - Option --delta works now nibble based instead of byte based.
 
  - Several small improvements.
 
  - Several small improvements.
 +
 +
 +
mkw-ana v0.11 r1622 - 2013-07-14
 +
 +
- More record types are detected now, some only as unnamed hex-records.
 +
- The 'PREFIX*' records are renamed to 'ROOM*', because they are only send in
 +
  room events. For non room events, there exists the prefix record 'GLOBAL'.
 +
- Optimized event detection, including world wide races and battles.
 +
- A first battle stage detection works now. Battle records are different from
 +
  racing rcords and not analysed yet.
 +
- New command: COUNT: Count the record types. Together with option --verbose
 +
  it shows all defined record types with short and long names.
 +
- Event and race numbers are now incremented at begin of an event.
 +
- Track, driver, vehicle and engine names are now available in (us)english
 +
  and german without loading a BMG text file. The default output language is
 +
  english. If option --de is set, the default output language is switched to
 +
  german. However, a loaded BMG text file may override all names.
 +
- Bug fix: Daylight saving is now recognized.
 +
- The checksum of each UDP packet is now checked and invalid packets are
 +
  dropped silently. If option --checksum is set, the packets are not dropped.
 +
  This makes only sense for hex dumps. If option --checksum is set twice, the
 +
  checksums are not calculated and assumed to be correct.
 +
- Some minor printing and counting bugs fixed.
 +
- Windows only: Cygwin update to v1.7.20 2013-06-07.
 +
- Windows only: The needed Cygwin DLL files are now delivered too.
 
</pre></spoiler>
 
</pre></spoiler>
  
Line 134: Line 126:
 
<spoiler><pre>
 
<spoiler><pre>
  
−
mkw-ana v0.11/x86_64 r1622 -- Dirk Clemens -- 2013-07-14
+
mkw-ana v0.12/x86_64 r1644 -- Dirk Clemens -- 2013-08-19
 
--------------------------------------------------------
 
--------------------------------------------------------
  
Line 217: Line 209:
 
                     load of team files. Multiple usage is possible.
 
                     load of team files. Multiple usage is possible.
 
     --origin x,y,z  Define an alternative origin for positions.
 
     --origin x,y,z  Define an alternative origin for positions.
 +
  
 
Command specific options with common description:
 
Command specific options with common description:
−
 
  
 
     --adjust float  Adjust time stamps of the network dump by adding 'float'
 
     --adjust float  Adjust time stamps of the network dump by adding 'float'
Line 231: Line 223:
 
                     executing options --skip and --term.
 
                     executing options --skip and --term.
 
     --checksum      Normally, UDP packets with wrong checksums are dropped. If
 
     --checksum      Normally, UDP packets with wrong checksums are dropped. If
−
                     --checksum set, the checksums are calculated, but no
+
                     --checksum is set, the checksums are calculated, but no
−
                     pakcet is dropped. Some dumps will print a status info. If
+
                     packet is dropped. Some dumps will print a status info. If
 
                     set twice, checksums are never calculated and assumed to
 
                     set twice, checksums are never calculated and assumed to
 
                     be correct. --csum is a short cut.
 
                     be correct. --csum is a short cut.
Line 323: Line 315:
 
                     bits) is unchanged, a '-' is printed intead of a hex
 
                     bits) is unchanged, a '-' is printed intead of a hex
 
                     digit.
 
                     digit.
−
   -I --index ranges  Dump only bytes witn an index selected by the range list.
+
   -I --index ranges  Dump only bytes with an index selected by the range list.
 
                     This make the hex dump smaller especially for one-line
 
                     This make the hex dump smaller especially for one-line
 
                     dumps.
 
                     dumps.
 
                       The parameter is a comma separated list of INDEX,
 
                       The parameter is a comma separated list of INDEX,
 
                     INDEX1:, INDEX1:INDEX2 and INDEX#LENGTH elements.
 
                     INDEX1:, INDEX1:INDEX2 and INDEX#LENGTH elements.
 +
  -P --print list    Print only the specified columns of the output table. A
 +
                    comma separated list of keywords is expected: RANK, TIME,
 +
                    POS=X-POS+Y-POS+Z-POS, DIR, SPEED=3D-SPEED,H-SPEED,
 +
                    STATUS, DRIFT=D-COUNT+D-CHEAT, XDRIFT=DRIFT+D-MINTIME,
 +
                    ITEM=I-CHEAT, CHEAT=D-CHEAT,I-CHEAT TEST, FC, MINI-FC and
 +
                    NAME. Also available: CLEAR, MIN, DEFAULT, MAX and ALL.
  
 
     --min-race num  This is a statistic option: If a Grand Prix (single or
 
     --min-race num  This is a statistic option: If a Grand Prix (single or
Line 355: Line 353:
 
     --mii dir      Extract Miis to the already existing directory 'dir'.
 
     --mii dir      Extract Miis to the already existing directory 'dir'.
 
                     Existing Mii files will be overwritten.
 
                     Existing Mii files will be overwritten.
 +
 
</pre></spoiler>
 
</pre></spoiler>
  

Revision as of 12:05, 19 August 2013

Wiimms mkw-ana
File:Wiimms-SZS-Tools.png
Author: Wiimm
Operating Systems: Linux (i386,x86_64),
Windows (Cygwin).
Software Type: Mario Kart Wii network analyzer
File Formats: PCAP 2.4 (tcpdump), bzip2 compression, BMG (text).
Current Version: 0.12, 2013-08-19

The tools mkw-ana is a new project by Wiimm to analyze the network protocol of Mario Kart Wii.



In Progress
This article is managed by Wiimm, the developer, and will grow slowly. Feel free to edit typos and grammatical or speech issues.

Intention

In November 2012, Wiimm decided to analyze the network protocol of Mario Kart Wii. The main goal was to discover online cheaters. Another goal is to set up his own server if any time Nintendo will shut down its servers.

So the main feature is to dump the packets of a tcpdump (done by tcpdump or wireshark) in a user-friendly format. After first experiments, it becomes also a live racing statistic tool.

Terms and definition

Template:MKWii Network Terms

The Tool

Change log

mkw-ana v0.12 r1644 - 2013-08-19

 - Bug fix: For aligment, the Mii data in user records is 0x4c bytes long,
   2 bytes more than the Mii file size (0x4a=74).
 - Detection of new records: ITEM_CTRL, ITEM_LIST, ITEM_DATA.
 - First item cheat detection, only visible for command RACE. It produce
   false positives, but if the counter is >5, a cheater play with you.
 - Command RACE: New output with head line, speed and item cheat infos.
 - New option for command RACE: --print=list (-P list):
    Print only the specified columns of the output table. A comma separated
    list of keywords is expected: RANK, TIME, POS=X-POS+Y-POS+Z-POS, DIR,
    SPEED=3D-SPEED,H-SPEED, STATUS, DRIFT=D-COUNT+D-CHEAT,
    XDRIFT=DRIFT+D-MINTIME, ITEM=I-CHEAT, CHEAT=D-CHEAT,I-CHEAT TEST, FC,
    MINI-FC and NAME. Also available: CLEAR, MIN, DEFAULT, MAX and ALL.
Old change log

Show text


Built-in Help

Let's start with the built-in help as an overview about the tool:

Show text


General Description

The tool started as simple hex dumper reading network dumps in PCAP format. In the first phase of the tool, the textual dumps of wireshark and tcpdump were much better. But after only a few days, the tool learned to handle records, clients, users, friend codes and Miis. From this moment the tool was better to analyze the Mario Kart Wii traffic.

Now, mkw-ana split the traffic into records and scans some data to detect stages of the online meeting. Stages are for example room, prepare race, count down, racing and end of race. It is able to separate races into events (grand prix and team rand prix) and to calculate racing tables. Racing data can also be exported to support live statistics.

At the moment there are three different kinds of hexdumps. All 3 are able to dump in one line mode to have large tables. Tool less is here a very good tool for vertical and horizontal scrolling. The stages are includes into the dump as comment lines. The dumped records can be filters by sending, receiving, proxy, record types, stage types and packet length. It is also possible to select the dumped bytes by indices and ranges.

Another feature is, that mkw-ana can read comment files. If making videos of the dumped meetings, you can write such comment file. Each line starts with a timestamp followed by a comment. Virtual Dub is a good tool for this job. Then you must synchronize the comment file with the network dump. The start of the first game ("GO" in the video) is a very good point for synchronization. Here is an example of a comment file (in german): 

>2012-12-05 19:10:39.745 - 12:22.792
0:00:00.000 Video Start

0:12:22.792 Rennen 1.1, GO!
0:12:54.123 Tinti wird angekündigt, T=0:31.322
0:14:17.924 Power wird angekündigt, T=1:55.120
0:15:33.633 Blitz schlägt ein, T=1:35.326
...
Notes
  • The first line is the real time of the start of the race minus the video time stamp. This is the synchronisation. An synchronisation can be done multiple times.
  • The line with video timestamp and comment follow. The focus of the comment change as the point of interest.
  • The name of the comment file must be the same as the network dump, but it must have the extension ".info" instead of ".eth".

???

Dumping Data

Options

Download

You can find the latest and some old distributions here
Content
  • Binaries for:
    • Linux i386
    • Linux x86_64
    • Cygwin/Windows (Needed Cygwin[1] DLL files are delivered. Best is to install a Cygwin system).
  • Some scripts as examples.
  • Some BMG text examples.
  • Some doc files.
Sometimes I upload single tool updates for testers
And remember
It is an alpha distribution and and only minimal support will be given at the moment.

Capture the network data

First you must capture the network traffic of the Wii. Therefore you must redirect it to a PC running a capture software. There are 3 general ways to to this:

  • If you have a manageable switch, enable port mirroring and send all Wii traffic to a PC.
  • Use your PC as router.
  • Use old network hubs (not switches). A hub will mirror all traffic of all ports to all others; it's just a multi port repeater and will slow down your network.

Use a software like tcpdump or wireshark to capture the data. Best is to save the captured data directly to a file or to send it to other commands like mkw-ana for a live analysis.

It's also possible to save the data to a file and to make a live analysis at the same time. Use the following command pipe: 

tcpdump -w- -U -i eth1 host wii | tee save.dump | mkw-ana ...

It is important to filter the data by host ip_or_name, because foreign traffic interfere the wii traffic analysis and will have curious side effects.

If using wireshark, save the dump to a file and use the following command for a live analysis: 

mkw-ana --follow DUMPFILE ...

Live Statistics

mkw-ana can scan the network traffic in real time and can produce makedoc or php data files. Together with ssh and an cgi script, a live statistic is created. Live means that the tables are updates 2-5 seconds after the race have finished.

How it works

The whole job is done by 3 processes:

  1. First, you must capture the network traffic like described above. Then use one of the commands: 
    ... | mkw-ana log --md  DATAFILE
... | mkw-ana log --php DATAFILE
mkw-ana --follow DUMPFILE log --md  DATAFILE
mkw-ana --follow DUMPFILE log --php DATAFILE
  2. Each time, a new DATAFILE is written, it must be transferred to the web server. A script using shh, sftp scp or ftp within an endless loop will do this job automatically.
  3. Last not least, a CGI or PHP script running at the web server must read the data files to serve a html-page to the visitors.

To see, what live means, visit the live statistics on Wednesday or Thursday between 19:10 and 20:30 CET (Central European Time).

Links

References

  1. ↑ Cygwin, a Linux like environment for Windows.


Template:MKWii Network Protocol

Retrieved from "https://wiki.tockdom.com/w/index.php?title=Wiimms_mkw-ana_(tool)&oldid=46543"