Difference between revisions of "Wiimms SZS Tools"

From Custom Mario Kart
Jump to navigation Jump to search
(Wiimms SZS Tools v0.03a)
Line 1: Line 1:
 
== Introduction ==
 
== Introduction ==
  
'''Wiimms SZS Tools''' should be a set of different tools to manipulate SZS files.  All tools are command line tools and run without any interaction. The main goal is to run them in batch files and scripts to automate recurring jobs. At this moment there is only one tool:
+
'''Wiimms SZS Tools''' should be a set of different tools to manipulate SZS files.  All tools are command line tools and run without any interaction. The main goal is to run them in batch files and scripts to automate recurring jobs. There are two tools:
 
* '''wszst''' : Wiimms SZS Tool
 
* '''wszst''' : Wiimms SZS Tool
 
+
* '''wbmgt''' : Wiimms BMG Tool
A second tool to change the content of message files is planned.
 
 
Both tools together can be used to change the track names of all message files of all languages without interaction.
 
Both tools together can be used to change the track names of all message files of all languages without interaction.
  
Line 13: Line 12:
 
'''Last changes:'''
 
'''Last changes:'''
 
<pre>
 
<pre>
szs v0.01a r2392 - 2011-04-01
+
szs v0.03a r2437 - 2011-04-06
 
 
- Bug fix for "wstsz CREATE": Wrong calculation of parent directory link.
 
- New option: wstsz CREATE --align: Overwrite the default aligment (32=0x20).
 
- wstsz LIST: If --long is set three times debugging mode is enabled.
 
- wstsz LIST: If listing a directory, then a virtual SZS file is created
 
  (like CREATE do it) and listed.
 
- New command: wszst DIFF: Compare two sources (each SZS, U8 or directory)
 
  on file level and report mismatches for each sub file.
 
 
 
szs v0.02a r2417 - 2011-04-04
 
  
  - New option: --max-file-size: This options set the limit for input files.
+
  - wszst: Compression algorithm improved: The compression is now about 40%-50%
   The default value is 20 MiB.
+
  faster than before.
  - Option --dest: If the parameter is '-' the output is redirected to stdout.
+
- wszst: New option --compr=level: define a compression level between 0-9.
  - New tool: wbgmt: Wiimms BGM Tool
+
  Three clear text parameters accepted too: NONE=0, FAST=1 and BEST=9.
   Commands:
+
- wszst: New option --fast: This is a short cut for --compr=fast. Usually
    - LIST  : List a message file to proof the content.
+
   "CREATE --fast" needs about 1/4 of time, but creates ~25% larger SZS files.
    - PATCH  : Patch the entered text files.
+
  - New command: wszst CAT: Extract one or more sub files of SZS/U8 archives.
    - DECODE : Decode (and patch) the entered text files.
+
  The output is written to stdout. Multiple outputs are concatenated.
    - ENCODE : Encode (and patch) the entered text files.
+
  - New command: wbmgt DIFF: Compare two BMG (raw or text) file on string level
 +
   and report mismatches for each string.
 +
- wbmgt --patch: new mode: HIDE
 +
- wbmgt is able to read BMG files directly from a SZS file. Just use a
 +
  combined path as 'my.szs/message/Common.bmg'
 +
- new option: --ignore: Ignore non existing source files without warning.
 
</pre>
 
</pre>
  
Line 47: Line 41:
 
* '''wszst''' supports compressed and uncompressed SZS files. Uncompressed files are good for analysis. The format of each source file is detected independent of the file name.
 
* '''wszst''' supports compressed and uncompressed SZS files. Uncompressed files are good for analysis. The format of each source file is detected independent of the file name.
 
* '''wszst LIST''' lists the content of a SZS files and directories. It lists also extracted file systems to preview the creation process.
 
* '''wszst LIST''' lists the content of a SZS files and directories. It lists also extracted file systems to preview the creation process.
* '''wszst DECOMPRESS''' and '''wszst COMPRESS''' convert SZS files from and to compressed format.
+
* '''wszst DIFF''' compares 2 SZS files on file level. This comparing is independent of the compression and of file order.
 +
* '''wszst COMPRESS''' and '''wszst DECOMPRESS''' convert SZS files from and to compressed format.
 +
* '''wszst CREATE''' creates SZS files from directory structures. All files, but not files starting with a point, are included.
 
* '''wszst EXTRACT''' extract the directories and files to the local file system to make access very easy.  
 
* '''wszst EXTRACT''' extract the directories and files to the local file system to make access very easy.  
* '''wszst CREATE''' creates SZS files from directory structures. All files, but not files starting with a point, are included.
 
 
* Extrating a SZS file and creating again results in the same SZS file. Only the order of the files may be changed.
 
* Extrating a SZS file and creating again results in the same SZS file. Only the order of the files may be changed.
* '''wszst DIFF''' compares 2 SZS files on file level. This comparing is independent of the compression and of file order.
+
* '''wszst CAT''' allow to access one or more sub files within SZS files. The output of all sources is concatenated and written to stdout.
 +
 
  
 
=== wszst HELP ===
 
=== wszst HELP ===
Line 132: Line 128:
 
* '''wbmgt''' supports raw and text BMG files. '''Raw BMG''' is the file format needed by Nintendos software. '''Text BMG''' is a user readable and editable format. In each text export an little syntax documentation is included.
 
* '''wbmgt''' supports raw and text BMG files. '''Raw BMG''' is the file format needed by Nintendos software. '''Text BMG''' is a user readable and editable format. In each text export an little syntax documentation is included.
 
* When reading a file, '''wbmgt''' detect the format of each source file independent of the file name. This make the usage of raw and text BMG files interchangable.
 
* When reading a file, '''wbmgt''' detect the format of each source file independent of the file name. This make the usage of raw and text BMG files interchangable.
 +
* '''wbmgt''' may read BMG files directly from SZS files. Just enter the path and assume that the SZS file is a directory.
 
* '''wbmgt LIST''' lists the contained strings for a fast preview.
 
* '''wbmgt LIST''' lists the contained strings for a fast preview.
 +
* '''wbmgt DIFF''' compares the strings of 2 BMG files and reports the differences.
 
* '''wbmgt PATCH''' patches each source by using a list of patch files. There are several patching modes: REPLACE, INSERT, OVERWRITE, DELETE, MASK, EQUAL and NONEQUAL. Like every source, raw and text BMG files are accepted as patch.
 
* '''wbmgt PATCH''' patches each source by using a list of patch files. There are several patching modes: REPLACE, INSERT, OVERWRITE, DELETE, MASK, EQUAL and NONEQUAL. Like every source, raw and text BMG files are accepted as patch.
 
* '''wbmgt DECODE''' converts each source file into text mode. Patching while decoding is possible.
 
* '''wbmgt DECODE''' converts each source file into text mode. Patching while decoding is possible.

Revision as of 15:02, 6 April 2011

Introduction

Wiimms SZS Tools should be a set of different tools to manipulate SZS files. All tools are command line tools and run without any interaction. The main goal is to run them in batch files and scripts to automate recurring jobs. There are two tools:

  • wszst : Wiimms SZS Tool
  • wbmgt : Wiimms BMG Tool

Both tools together can be used to change the track names of all message files of all languages without interaction.

This toolset will not replace the SZS Modifier or SZS Explorer, because they are interactive tools with GUI support and can work directly with many sub file formats.

History and Download

Last changes:

szs v0.03a r2437 - 2011-04-06

 - wszst: Compression algorithm improved: The compression is now about 40%-50%
   faster than before.
 - wszst: New option --compr=level: define a compression level between 0-9.
   Three clear text parameters accepted too: NONE=0, FAST=1 and BEST=9.
 - wszst: New option --fast: This is a short cut for --compr=fast. Usually
   "CREATE --fast" needs about 1/4 of time, but creates ~25% larger SZS files.
 - New command: wszst CAT: Extract one or more sub files of SZS/U8 archives.
   The output is written to stdout. Multiple outputs are concatenated.
 - New command: wbmgt DIFF: Compare two BMG (raw or text) file on string level
   and report mismatches for each string.
 - wbmgt --patch: new mode: HIDE
 - wbmgt is able to read BMG files directly from a SZS file. Just use a
   combined path as 'my.szs/message/Common.bmg'
 - new option: --ignore: Ignore non existing source files without warning.

wszst (Wiimms SZS Tool)

wszst is a command driven tool to manipulate SZS files.

Features

  • wszst supports compressed and uncompressed SZS files. Uncompressed files are good for analysis. The format of each source file is detected independent of the file name.
  • wszst LIST lists the content of a SZS files and directories. It lists also extracted file systems to preview the creation process.
  • wszst DIFF compares 2 SZS files on file level. This comparing is independent of the compression and of file order.
  • wszst COMPRESS and wszst DECOMPRESS convert SZS files from and to compressed format.
  • wszst CREATE creates SZS files from directory structures. All files, but not files starting with a point, are included.
  • wszst EXTRACT extract the directories and files to the local file system to make access very easy.
  • Extrating a SZS file and creating again results in the same SZS file. Only the order of the files may be changed.
  • wszst CAT allow to access one or more sub files within SZS files. The output of all sources is concatenated and written to stdout.


wszst HELP

The HELP command print the built in help. Type wszst HELP for a general help. Add a command to print command specific help:

wszst LIST

The command LIST list the sub files of SZS Files.

Example:

# wszst lll pool/ref/szs/Award_G.szs

* Files of pool/ref/szs/Award_G.szs

off/hex siz/hex size/dec  file or directory/
-------------------------------------------------------------------------------
      -       -        -  ./
      -       -        -  ./award/
      -       -        -  ./dpd_pointer/
      -       -        -  ./message/
    120    2580     9600  ./message/Common.bmg
   26a0   18300    99072  ./message/Menu.bmg
  1a9a0    1fc0     8128  ./message/Race.bmg
      -       -        -  ./message_window/
      -       -        -  ./pad_recognize/
      -       -        -  ./winning_run/

wszst DIFF

The command DIFF compares two sources (each SZS, U8 or directory) on file level and report mismatches for each sub file.

Example:

# wszst diff pool/ref/szs/Award_G.szs pool/ref/szs/MenuMulti_G.szs
* Only in source #1: ./award/
* Only in source #2: ./bg/
* Only in source #2: ./button/
* Only in source #2: ./control/
* Only in source #2: ./globe/
* Only in source #1: ./message/Race.bmg
* Only in source #2: ./model/
Content differ: pool/ref/szs/Award_G.szs : pool/ref/szs/MenuMulti_G.szs

wszst DECOMPRESS

The command DECOMPRESS decompresses a SZS File and stores the decompressed data into file.

wszst COMPRESS

The command COMPRESS compresses a binary file and creates SZS File.

wszst EXTRACT

The command EXTRACT extracts the sub file of source files and copy the files into a directory structure.

wszst CREATE

The command CREATE creates a SZS file by adding all files of the directory structure.

wbmgt (Wiimms BMG Tool)

wbmgt is a command driven tool to manipulate BMG files.

Features

  • wbmgt supports raw and text BMG files. Raw BMG is the file format needed by Nintendos software. Text BMG is a user readable and editable format. In each text export an little syntax documentation is included.
  • When reading a file, wbmgt detect the format of each source file independent of the file name. This make the usage of raw and text BMG files interchangable.
  • wbmgt may read BMG files directly from SZS files. Just enter the path and assume that the SZS file is a directory.
  • wbmgt LIST lists the contained strings for a fast preview.
  • wbmgt DIFF compares the strings of 2 BMG files and reports the differences.
  • wbmgt PATCH patches each source by using a list of patch files. There are several patching modes: REPLACE, INSERT, OVERWRITE, DELETE, MASK, EQUAL and NONEQUAL. Like every source, raw and text BMG files are accepted as patch.
  • wbmgt DECODE converts each source file into text mode. Patching while decoding is possible.
  • wbmgt ENCODE converts each source file into raw mode. Patching while encoding is possible.
  • Decoding a raw BMG file and encoding again results in the same BMG file.

Tests

I have made several tests with the following files:

  • All files of Scene/UI/.
  • All files of Race/Course/, some of the tracks have been replaced by custom tracks before.

The tests:

  • Decompress and compress and decompress again SZS files and compare the results.
  • Extract all files, create SZS and extract again. The both extracted file system are equal.
  • Extract some of the files with SZS Explorer and compare them to my extracted files.
  • Extract all files and create a new SZS files. Then create a new ISO image with the new SZS files and play it.
  • Read BMG file, convert to internal representation and write a new BMG. Then compare the new BMG with the original one.
  • Convert all BMG files into the text format and back again. Then compare the new BMG with the original one.

All tests are run without errors.

Links