Difference between revisions of "Wiimms SZS Tools"

From Custom Mario Kart
Jump to navigation Jump to search
(v0.34a)
(new layout)
Line 27: Line 27:
 
== Introduction ==
 
== Introduction ==
  
'''Wiimms SZS Tools''' is a set of different tools to manipulate [[SZS]], [[U8]], [[BRRES]] and [[BMG]] files. All tools are command line tools and run without any interaction. The tools are available for Linux, mac and Windows. The main goal is to run them in batch files and scripts to automate recurring jobs. At the moment there are two tools:
+
'''Wiimms SZS Tools''' is a set of different tools to manipulate [[SZS]], [[U8]], [[RARC]], [[BRRES]], [[BREFF]] and [[BREFT]] archives and [[BMG]], [[BTI]], [[KCL]], [[KMP]], [[MDL0]], [[TPL]], [[TEX0]] and [[StaticR.rel]] files. All tools are command line tools and run without any interaction. The tools are available for Linux, Mac and Windows. The main goal is to run them in batch files and scripts to automate recurring jobs. At the moment there are 6 tools:
* '''wszst''' : Wiimms SZS Tool (handles [[SZS]], [[U8]], [[BRRES]], [[BREFF]] and [[BREFT]] archives)
+
* '''wszst''' : Wiimms SZS Tool (handles [[SZS]], [[U8]], [[RARC]], [[BRRES]], [[BREFF]] and [[BREFT]] archives)
* '''wbmgt''' : Wiimms BMG Tool (handles raw and text [[BMG]] files).
+
* '''wbmgt''' : Wiimms BMG Tool (convert and patch [[BMG]] files in binary and text format).
* '''wimgt''' : Wiimms Image Tool (convert image files of [[Mario Kart Wii]]).
+
* '''wimgt''' : Wiimms Image Tool (convert and patch image files of [[Mario Kart Wii]]).
 
* '''wkclt''' : Wiimms KCL Tool (decode only [[KCL]] files into text/obj format).
 
* '''wkclt''' : Wiimms KCL Tool (decode only [[KCL]] files into text/obj format).
* '''wkmpt''' : Wiimms KMP Tool (handles raw and text [[KMP]] files, convert text files like a compiler).
+
* '''wkmpt''' : Wiimms KMP Tool (handles raw and text [[KMP]] files, read text files like a compiler).
 
* '''wstrt''' : Wiimms StaticR Tool (handles [[StaticR.rel]] files of [[Mario Kart Wii]]).
 
* '''wstrt''' : Wiimms StaticR Tool (handles [[StaticR.rel]] files of [[Mario Kart Wii]]).
 
One main goal is the fully automated [[Wiimms SZS Tools/How To#Replace Track Names|track name replacement]]. The tools together can do this. Another goal is to extract all sub files recursivley and transforming they into user friendly file formats (like text of png files) and support recreation of the original source files.
 
One main goal is the fully automated [[Wiimms SZS Tools/How To#Replace Track Names|track name replacement]]. The tools together can do this. Another goal is to extract all sub files recursivley and transforming they into user friendly file formats (like text of png files) and support recreation of the original source files.
 
This toolset will not replace the [[SZS Modifier]] or the [[CTools]], because they are interactive tools with GUI support and can work directly with many sub file formats.
 
  
 
'''There is also an little [[Wiimms SZS Tools/How To|How To]].'''
 
'''There is also an little [[Wiimms SZS Tools/How To|How To]].'''
  
 
== History and Download ==
 
== History and Download ==
 
=== What's new ===
 
 
* Support of [[BTI]] images, see change log for details.
 
  
 
=== Change Log ===
 
=== Change Log ===
Line 114: Line 108:
 
* [http://szs.wiimm.de/cgi/mkw/object KMP Object Query]
 
* [http://szs.wiimm.de/cgi/mkw/object KMP Object Query]
  
== wszst (Wiimms SZS Tool) ==
+
== Tools Overview ==
 
 
'''wszst''' is a command driven tool to manipulate [[SZS]] files.
 
* [http://szs.wiimm.de/wszst Tool wszst: Features, commands and options]
 
 
 
=== Features ===
 
 
 
* '''wszst''' supports compressed and uncompressed [[SZS]] files. Uncompressed files are good for analysis. Supported archive formats are [[U8]], [[BRRES]], [[BREFF]] and [[BREFT]]. The format of each source file is detected independent of the file name.
 
* '''wszst LIST''' lists the content of a [[SZS]], [[U8]], [[BRRES]], [[BREFF]] and [[BREFT]] archives and directories. It lists also extracted file systems to preview the creation process.
 
* '''wszst DIFF''' compares 2 SZS files on file level. This comparing is independent of the compression and of file order.
 
* '''wszst NORMALIZE''' read an [[U8]] archive, compressed or not, and write it back in a normalized form.
 
** The files are sorted in a ''Nintendo like'' order.
 
** Unused data holes are removed.
 
** If directory <tt>'.'</tt> is available it becomes the root directory for all others.
 
** Compressed sources are compressed again with maximum compression (level 9).
 
** The destination is only written if any data changed.
 
* '''wszst COMPRESS''' and '''wszst DECOMPRESS''' convert SZS files from and to compressed format.
 
* '''wszst EXTRACT''' extract the directories and files to the local file system to make access very easy.  Recursive extraction and automatic decoding of some file formats is also possible.
 
* '''wszst EXTRACT''' extract known archives reclusively and can decode BMG files and graphic images on the fly.
 
* '''wszst CREATE''' creates SZS files from directory structures. All files are included, include and exlcude lists are supported. Recursive creation and automatic encoding of some file formats is also possible.
 
* '''wszst CAT''' allow to access one or more sub files within SZS files. The output of all sources is concatenated and written to stdout.
 
* Extrating a [[SZS]] file and creating again results in the same SZS file. Only the internal order of the files may be changed.
 
* '''wszst DIFF''' compares 2 SZS files on file level. This comparing is independent of the compression and of file order.
 
 
 
=== wszst LIST ===
 
 
 
The command '''LIST''' list the sub files of [[SZS]] files.
 
 
 
&rArr; [http://szs.wiimm.de/cmd/wszst/list wszst LIST]
 
 
 
'''Example:'''
 
<pre>
 
# wszst lll Award_G.szs
 
 
 
* Files of Award_G.szs
 
 
 
off/hex siz/hex size/dec  magic file or directory/
 
-------------------------------------------------------------------------------
 
      -      -        -  -    ./
 
      -      -        -  -    ./award/
 
      -      -        -  -    ./dpd_pointer/
 
      -      -        -  -    ./message/
 
    120    2580    9600  MESG  ./message/Common.bmg
 
  26a0  18300    99072  MESG  ./message/Menu.bmg
 
  1a9a0    1fc0    8128  MESG  ./message/Race.bmg
 
      -      -        -  -    ./message_window/
 
      -      -        -  -    ./pad_recognize/
 
      -      -        -  -    ./winning_run/
 
</pre>
 
 
 
=== wszst DIFF ===
 
 
 
The command '''DIFF''' compares two sources (each [[SZS]], [[U8]], [[BRRES]] or directory)
 
on file level and report mismatches for each sub file.
 
 
 
&rArr; [http://szs.wiimm.de/cmd/wszst/diff wszst DIFF]
 
 
 
'''Example:'''
 
<pre>
 
# wszst diff Award_G.szs MenuMulti_E.szs
 
 
 
* Only in source #1: ./award/
 
* Only in source #2: ./bg/
 
* Only in source #2: ./button/
 
* Only in source #2: ./control/
 
* Only in source #2: ./globe/
 
* File size differ:  ./message/Common.bmg [9984+256=10240]
 
* File size differ:  ./message/Menu.bmg [97920-7552=90368]
 
* Only in source #1: ./message/Race.bmg
 
* Only in source #2: ./model/
 
* Only in source #1: ./winning_run/
 
Content differ: Award_G.szs : MenuMulti_E.szs
 
</pre>
 
 
 
=== wszst NORMALIZE ===
 
 
 
The command '''NORMALIZE''' read an [[U8]] archive, compressed or not,
 
and write it back in a normalized form:
 
* The files are sorted in a ''Nintendo like'' order.
 
** ('/' is less than) '.' is less than digits is less than letters is less than others.
 
** The case of letters is ignored.
 
** For same group of characters use ASCII order.
 
** For each directory: Real files are inserted before sub directories.
 
* Unused data holes are removed.
 
* If directory <tt>'.'</tt> is available it becomes the root directory for all others.
 
* Compressed sources are compressed again with maximum compression (level 9).
 
* The destination is only written if any data changed.
 
 
 
Normalized files are for archives and to find files with the same content.
 
 
 
&rArr; [http://szs.wiimm.de/cmd/wszst/normalize wszst NORMALIZE]
 
 
 
== wbmgt (Wiimms BMG Tool) ==
 
 
 
'''wbmgt''' is a command driven tool to manipulate [[BMG]] files.
 
* [http://szs.wiimm.de/wbmgt Tool wbmgt: Features, commands and options]
 
 
 
=== Features ===
 
 
 
* '''wbmgt''' supports raw and text BMG files. '''Raw BMG''' is the file format needed by Nintendos software. '''Text BMG''' is a user readable and editable format. '''wbmgt''' includes in every text export a little syntax documentation.
 
* When reading a file, '''wbmgt''' detect the format of each source file independent of the file name. This make the usage of raw and text BMG files interchangeable.
 
* '''wbmgt''' may read BMG files directly from SZS files. Just enter the path and assume that the SZS file is a directory. Example: <tt>path/to/my.szs/message/Common.bmg</tt>
 
* '''wbmgt LIST''' lists the contained strings for a fast preview.
 
* '''wbmgt DIFF''' compares the strings of 2 BMG files and reports the differences.
 
* '''wbmgt PATCH''' patches each source by using a list of patch files. There are several patching modes: REPLACE, INSERT, OVERWRITE, DELETE, MASK, EQUAL and NONEQUAL. Like every source, raw and text BMG files are accepted as patch.
 
* '''wbmgt DECODE''' converts each source file into text mode. Patching while decoding is possible.
 
* '''wbmgt ENCODE''' converts each source file into raw mode. Patching while encoding is possible.
 
* Decoding a raw BMG file and encoding again results in the same BMG file.
 
  
== wimgt (Wiimms Image Tool) ==
+
=== wszst (Wiimms SZS Tool) ===
  
'''wimgt''' is a command driven tool to manipulate graphic images of [[Mario Kart Wii]].
+
'''wszst''' is a command driven tool to manipulate archives:
* [http://szs.wiimm.de/wimgt Tool wimgt: Features, commands and options]
+
* Supported archive formats: [[SZS]], [[U8]], [[RARC]], [[BRRES]], [[BREFF]] and [[BREFT]].
 +
* It can extract all subfiles of any archive.
 +
* While extracting some file formats can be decoded (converted to text or PNG files).
 +
* It can create all (but not [[RARC]] archives) while readind and scanning a directory structure. Decoded files are encoded autoamtically.
 +
&rarr; [http://szs.wiimm.de/wszst Tool wszst: Features, commands and options]
  
=== Features ===
+
=== wbmgt (Wiimms BMG Tool) ===
  
* '''wimgt''' supports internal image formats (I4, I8, IA4, IA8, RGB565, RGB5A3, RGBA32, C4, C8, C14X2 and CMPR) and PNG files. '''wimgt''' and also '''wszst EXTRACT''' will search for images in BREFT subfiles, TPL and TEX0 files.
+
'''wbmgt''' is a command driven tool to manipulate [[BMG]] (message) files.
* When reading a file, '''wimgt''' detect the image format of each source file independent of the file name.
+
<br/>&rarr; [http://szs.wiimm.de/wbmgt Tool wbmgt: Features, commands and options]
* '''wimgt''' may read images directly from SZS files. Just enter the path and assume that the SZS file is a directory.
 
* '''wimgt LIST''' lists the images of a director and print out geometry data for each image.
 
* '''wimgt DECODE''' converts each image into a PNG file.
 
  
== wkmpt (Wiimms KMP Tool) ==
+
=== wkmpt (Wiimms KMP Tool) ===
  
'''wkmpt''' is a command driven tool to convert [[KMP]] files into/from binary from/into text files.
+
'''wkmpt''' is a command driven tool to convert [[KMP]] files into/from binary from/into text files. A special compiler is used to allow variables, expressions, if..then..else, loops, macros and user defined functions while reading a KMP text file.
* [http://szs.wiimm.de/wkmpt Tool wkmpt: Features, commands and options]
+
<br/>&rarr; [http://szs.wiimm.de/wkmpt Tool wkmpt: Features, commands and options]
* [[Wiimms SZS Tools/KMP|How to edit KMP files]]
+
<br/>&rarr; [http://szs.wiimm.de/doc/syntax General parser syntax and semantics]
 +
<br/>&rarr; [http://szs.wiimm.de/doc/kmp/syntax KMP text syntax and semantics]
 +
<br/>&rarr; [http://szs.wiimm.de/doc/kmp/func KMP parser functions]
 +
<br/>&rarr; [[Wiimms SZS Tools/KMP|How to edit KMP files]]
  
=== Features ===
+
=== wimgt (Wiimms Image Tool) ===
  
* '''wkmpt''' supports raw and text KMP files. ''Raw KMP'' is the file format needed by Nintendos software. ''Text KMP'' is a user readable and editable format. Each text export contains a syntax documentation.
+
'''wimgt''' is a command driven tool to convert and aptch graphic images:
* When reading a file, '''wkmpt''' detect the format of each source file independent of the file name. This make the usage of raw and text KMP files interchangeable.
+
* '''wimgt''' supports the file formats [[BREFT]], [[BTI]], [[TEX0]], [[TPL]] and [[PNG]] as export and import type.
* '''wkmpt''' may read KMP files directly from SZS files. Just enter the path and assume that the SZS file is a directory.
+
* '''wimgt''' supports the internal image formats I4, I8, IA4, IA8, RGB565, RGB5A3, RGBA32, C4, C8, C14X2 and CMPR (and also different PNG formats).
** Example, where <tt>path/to/my.szs</tt> is the SZS file: <tt>path/to/my.szs/course.kmp</tt>
+
&rarr; [http://szs.wiimm.de/wimgt Tool wimgt: Features, commands and options]
* '''wkmpt DECODE''' converts each source file into text mode. These text files can be edited with any text editor and it's complete power. Also script or other text tools may manipulated these text files.
 
* '''wkmpt ENCODE''' converts each source file into raw mode. Indexing and grouping are done automatically. Some other automatic calculations can be enabled by switches within the text files.
 
* To make reorganisation (inserting, deleting and reordering entries) easier, the [[Wiimms SZS Tools/KMP|KMP text scanner]] supports global variables for inter sections links (CAME+GOBJ to POTI, CKPT to JGPT).
 
* Decoding a raw KMP file and encoding again results in the same KMP file. The only difference may be floating point numbers because of rounding of the least significant bits.
 
  
== wstrt (Wiimms StaticR Tool) ==
+
=== wstrt (Wiimms StaticR Tool) ===
  
 
'''wstrt''' is a command driven tool to manipulate [[StaticR]] files of [[Mario Kart Wii]].
 
'''wstrt''' is a command driven tool to manipulate [[StaticR]] files of [[Mario Kart Wii]].
* [http://szs.wiimm.de/wstrt Tool wstrt: Features, commands and options]
+
<br/>&rarr; [http://szs.wiimm.de/wstrt Tool wstrt: Features, commands and options]
 
 
=== Features ===
 
 
 
* '''wstrt''' can analyze and patch StaticR files of Mario Kart Wii (PAL and NTSC).
 
* '''wstrt ANALYZE''' analyzes StaticR files and detects CTGP 4.4 patches and modifications of track and arena orders. A possible patch is virtually applied before printing.
 
* '''wstrt PATCH''' modifies the track and arena order of StaticR files. So it is possible to move tracks with special features (like snow or fog) to other slots.
 
* '''wstrt TRACKS''' and '''wstrt ARENAS''' print a list of tracks or arenas with slots and filenames (track and sound files). If a StaticR file is entered, the track/arena order of that StaticR is used. A possible patch is virtually applied before printing.
 
* '''wstrt FILES''' prints a list of filenames of all tracks and arenas in machine readable format. If a StaticR file is entered, the track and arena order of that StaticR is used. A possible patch is virtually applied before printing.
 
  
 
== Links ==
 
== Links ==
Line 271: Line 150:
 
** [[Wiimms SZS Tools/KMP|KMP How To]]
 
** [[Wiimms SZS Tools/KMP|KMP How To]]
 
* [http://szs.wiimm.de/ Wiimms SZS Tools]
 
* [http://szs.wiimm.de/ Wiimms SZS Tools]
** [http://szs.wiimm.de/wszst Tool wszst: Features, commands and options]
+
** Features, commands and options: [http://szs.wiimm.de/wszst wszst], [http://szs.wiimm.de/wbmgt wbmgt],  [http://szs.wiimm.de/wkmpt wkmpt],  [http://szs.wiimm.de/wimgt wimgt], [http://szs.wiimm.de/wstrt wstrt],
** [http://szs.wiimm.de/wbmgt Tool wbmgt: Features, commands and options]
 
 
** [http://szs.wiimm.de/download.html Downloads]
 
** [http://szs.wiimm.de/download.html Downloads]
* [http://www.mariokartwii.com/f72/wiimms-szs-tools-75689.html Discussion at MarioKartWii.com]
+
* [[KMP]] Parser
* [http://gbatemp.net/t286476-wiimms-szs-tools?view=findpost&p=3559819 Discussion at GBAtemp.net]
+
** [http://szs.wiimm.de/doc/syntax General parser syntax and semantics]
* [http://forum.wii-homebrew.com/board197-pc-tools/p319981-wiimms-szs-tools/#post319981 Discussion at Wii-Homebrew.com] (german forum)
+
** [http://szs.wiimm.de/doc/kmp/syntax KMP text syntax and semantics]
 
+
** [http://szs.wiimm.de/doc/kmp/func KMP parser functions]
 +
* Forums
 +
** [http://www.mariokartwii.com/f72/wiimms-szs-tools-75689.html Discussion at MarioKartWii.com]
 +
** [http://gbatemp.net/t286476-wiimms-szs-tools?view=findpost&p=3559819 Discussion at GBAtemp.net]
 +
** [http://forum.wii-homebrew.com/board197-pc-tools/p319981-wiimms-szs-tools/#post319981 Discussion at Wii-Homebrew.com] (german forum)
  
 
{{Custom Track Tutorial}}
 
{{Custom Track Tutorial}}
 
{{User-Wiimm-Link}}
 
{{User-Wiimm-Link}}
 
[[category:Software]]
 
[[category:Software]]

Revision as of 13:35, 12 May 2012

Wiimms SZS Tools
File:Wiimms-SZS-Tools.png
Author: Wiimm
Operating Systems: Linux (i386,x86_64),
Mac (ppc,i386,x86_64),
Windows (cygwin).
Software Type: SZS, BMG, KMP, StaticR.rel and Image Manager
Archive Formats: SZS, U8, RARC, BRRES, BREFF, BREFT.
File Formats: BMG, BTI, KCL, KMP, MDL0, TPL, TEX0, StaticR.rel, PNG.
Current Version: 0.34a, 2012-05-12
Website: szs.wiimm.de

Introduction

Wiimms SZS Tools is a set of different tools to manipulate SZS, U8, RARC, BRRES, BREFF and BREFT archives and BMG, BTI, KCL, KMP, MDL0, TPL, TEX0 and StaticR.rel files. All tools are command line tools and run without any interaction. The tools are available for Linux, Mac and Windows. The main goal is to run them in batch files and scripts to automate recurring jobs. At the moment there are 6 tools:

  • wszst : Wiimms SZS Tool (handles SZS, U8, RARC, BRRES, BREFF and BREFT archives)
  • wbmgt : Wiimms BMG Tool (convert and patch BMG files in binary and text format).
  • wimgt : Wiimms Image Tool (convert and patch image files of Mario Kart Wii).
  • wkclt : Wiimms KCL Tool (decode only KCL files into text/obj format).
  • wkmpt : Wiimms KMP Tool (handles raw and text KMP files, read text files like a compiler).
  • wstrt : Wiimms StaticR Tool (handles StaticR.rel files of Mario Kart Wii).

One main goal is the fully automated track name replacement. The tools together can do this. Another goal is to extract all sub files recursivley and transforming they into user friendly file formats (like text of png files) and support recreation of the original source files.

There is also an little How To.

History and Download

Change Log

szs v0.34a r3659 - 2012-05-12

 - KMP text parser extensions:

    - New commands:
       @INCLUDE "filename": include text files.
       @RETURN [val]: returns from the current input file.
       @EXIT: exit the parser execution.
       @ASSERT condition: exit execution, if 'condition' is false.
       @MACRO .. @ENDMACRO: Define named macros.
       @FUNCTION .. @ENDFUNCTION: Like @MACRO, but can also be used as
          function in numerical expressions.
       @CALL macroname(...): Call a previos defined macro with parameters.
          '@:macroname(...)' is a short cut for '@CALL macroname(...):'
       @DOIF (condition) anytext: Execute 'anytext', if 'condition' is true.

    - New private variable name space: @PDEF
       - P-Variables are only defined for the current source file or function.
       - Loop variables (@FOR,@FOREACH) and macro parameters are defined in
         this new private name space.

 - New options for command FUNCTIONS (list parser functions):
    - Option --long: The entered keys are compared against the description.
    - Option --verbose: The description is printed too.

 - Bug fix: Parser function minMax() returned wrong results.


szs v0.33a r3627 - 2012-05-02

 - Because of an update of the internal object & file database, redo the
   command "wszst AUTOADD ./path_to/Race/Course/" to add more files to the
   autoadd database -> see http://szs.wiimm.de/doc/autoadd for more info.

 - wszst CHECK: Detect "optional files".

 - KMP: New command '@HSNAP-TO-ENPT' is similar to '@SNAP-TO-ENPT', but
   ignoring the vertical value of the enemy points.

 - The text parser supports 5 kinds of nested loops (EXPERIMENTAL):
     @LOOP    ... @ENDLOOP   : umlimted loop
     @REPEAT  ... @ENDREPEAT : loop with a predefined repeat count
     @FOR     ... @ENDFOR    : classical 'for' with iterator variable
     @FOREACH ... @ENDEACH   : for each element in list
     @WHILE   ... @ENDWHILE  : classical 'while' with condition
   There are also conditional @BREAK and @CONTINUE.

 - The KMP syntax and semantics documentation is now online available:
    -> http://szs.wiimm.de/doc/kmp/syntax
   The inline documentation in decoded KMP files is now much shorter.

 - Command SYNTAX removed. Use page http://szs.wiimm.de/doc/syntax instead.

 - The backtracking algorithm for analyzing of CKPH + ENPH + ITPH pathes has
   been modified and optimized, because a test version of "Toad's Turnpike"
   has 105 ITPH groups and needs much to much steps and time to calculate
   about 10^28 pathes total.

Links

Tools Overview

wszst (Wiimms SZS Tool)

wszst is a command driven tool to manipulate archives:

  • Supported archive formats: SZS, U8, RARC, BRRES, BREFF and BREFT.
  • It can extract all subfiles of any archive.
  • While extracting some file formats can be decoded (converted to text or PNG files).
  • It can create all (but not RARC archives) while readind and scanning a directory structure. Decoded files are encoded autoamtically.

Tool wszst: Features, commands and options

wbmgt (Wiimms BMG Tool)

wbmgt is a command driven tool to manipulate BMG (message) files.
Tool wbmgt: Features, commands and options

wkmpt (Wiimms KMP Tool)

wkmpt is a command driven tool to convert KMP files into/from binary from/into text files. A special compiler is used to allow variables, expressions, if..then..else, loops, macros and user defined functions while reading a KMP text file.
Tool wkmpt: Features, commands and options
General parser syntax and semantics
KMP text syntax and semantics
KMP parser functions
How to edit KMP files

wimgt (Wiimms Image Tool)

wimgt is a command driven tool to convert and aptch graphic images:

  • wimgt supports the file formats BREFT, BTI, TEX0, TPL and PNG as export and import type.
  • wimgt supports the internal image formats I4, I8, IA4, IA8, RGB565, RGB5A3, RGBA32, C4, C8, C14X2 and CMPR (and also different PNG formats).

Tool wimgt: Features, commands and options

wstrt (Wiimms StaticR Tool)

wstrt is a command driven tool to manipulate StaticR files of Mario Kart Wii.
Tool wstrt: Features, commands and options

Links