Difference between revisions of "Wiimms SZS Tools"

From Custom Mario Kart
Jump to navigation Jump to search
Line 35: Line 35:
 
== wszst (Wiimms SZS Tool) ==
 
== wszst (Wiimms SZS Tool) ==
  
'''wszst''' is a command driven tool to manipulate [[SZS file]]s.
+
'''wszst''' is a command driven tool to manipulate [[SZS]] files.
  
 
=== Features ===
 
=== Features ===
Line 41: Line 41:
 
* '''wszst LIST''' lists the content of a SZS files and directories. It lists also extracted file systems to preview the creation process.
 
* '''wszst LIST''' lists the content of a SZS files and directories. It lists also extracted file systems to preview the creation process.
 
* '''wszst DIFF''' compares 2 SZS files on file level. This comparing is independent of the compression and of file order.
 
* '''wszst DIFF''' compares 2 SZS files on file level. This comparing is independent of the compression and of file order.
* '''wszst NORMALIZE''' read an U8 archive, compressed or not, and write it back in a normalized form.
+
* '''wszst NORMALIZE''' read an [[U8]] archive, compressed or not, and write it back in a normalized form.
 
* '''wszst COMPRESS''' and '''wszst DECOMPRESS''' convert SZS files from and to compressed format.
 
* '''wszst COMPRESS''' and '''wszst DECOMPRESS''' convert SZS files from and to compressed format.
 
* '''wszst CREATE''' creates SZS files from directory structures. All files, but not files starting with a point, are included.
 
* '''wszst CREATE''' creates SZS files from directory structures. All files, but not files starting with a point, are included.
Line 57: Line 57:
 
=== wszst LIST ===
 
=== wszst LIST ===
  
The command '''LIST''' list the sub files of [[SZS File]]s.
+
The command '''LIST''' list the sub files of [[SZS]] files.
  
 
⇒ [http://szs.wiimm.de/cmd/wszst/list wszst LIST]
 
⇒ [http://szs.wiimm.de/cmd/wszst/list wszst LIST]
Line 83: Line 83:
 
=== wszst DIFF ===
 
=== wszst DIFF ===
  
The command '''DIFF''' compares two sources (each SZS, U8 or directory)
+
The command '''DIFF''' compares two sources (each [[SZS]], [[U8]] or directory)
 
on file level and report mismatches for each sub file.
 
on file level and report mismatches for each sub file.
  
Line 103: Line 103:
 
=== wszst NORMALIZE ===
 
=== wszst NORMALIZE ===
  
The command '''NORMALIZE''' read an U8 archive, compressed or not,
+
The command '''NORMALIZE''' read an [[U8]] archive, compressed or not,
 
and write it back in a normalized form:
 
and write it back in a normalized form:
 
* The files are sorted in ASCII order.
 
* The files are sorted in ASCII order.
Line 114: Line 114:
 
=== wszst COMPRESS ===
 
=== wszst COMPRESS ===
  
The command '''COMPRESS''' compresses a binary file and creates [[SZS File]].
+
The command '''COMPRESS''' compresses a binary file and creates [[SZS]] file.
  
 
⇒ [http://szs.wiimm.de/cmd/wszst/compress wszst COMPRESS]
 
⇒ [http://szs.wiimm.de/cmd/wszst/compress wszst COMPRESS]
Line 120: Line 120:
 
=== wszst DECOMPRESS ===
 
=== wszst DECOMPRESS ===
  
The command '''DECOMPRESS''' decompresses a [[SZS File]]
+
The command '''DECOMPRESS''' decompresses a [[SZS]] file
 
and stores the decompressed data into file.
 
and stores the decompressed data into file.
  
Line 127: Line 127:
 
=== wszst CREATE ===
 
=== wszst CREATE ===
  
The command '''CREATE''' creates a [[SZS file]]
+
The command '''CREATE''' creates a [[SZS]] file
 
by adding all files of the directory structure.  
 
by adding all files of the directory structure.  
  
Line 141: Line 141:
 
=== wszst CAT ===
 
=== wszst CAT ===
  
The command '''CAT''' one or more named sub files of SZS and U8 archives.
+
The command '''CAT''' one or more named sub files of [[SZS]] and [[U8]] archives.
 
The output of all sources is concatenated and written to stdout.
 
The output of all sources is concatenated and written to stdout.
  

Revision as of 13:59, 10 April 2011

Introduction

Wiimms SZS Tools is a set of different tools to manipulate SZS and BMG files. All tools are command line tools and run without any interaction. The main goal is to run them in batch files and scripts to automate recurring jobs. There are two tools:

  • wszst : Wiimms SZS Tool
  • wbmgt : Wiimms BMG Tool

Both tools together can be used to change the track names of all message files of all languages without interaction. One main goal is the fully automated track name replacement.

This toolset will not replace the SZS Modifier or SZS Explorer, because they are interactive tools with GUI support and can work directly with many sub file formats.

History and Download

Last changes:

szs v0.04a r2457 - 2011-04-09

 - SZS compression improved: It uses now overlaps and save about 2% size.
   With more complicated algorithm (that needs much more time) better
   compression is possible -- but it's not worth it.
 - wszst LIST --long: View the magic (first 4 bytes) of each file too.
 - wszst can list, diff, (de-)compress and extract SZS.BRRES files.
 - wbgmt: Bug fix: Wrong string end detection for the last message if
   scanning Nintendos 0x1a (CTRL-Z, ASCII 'SUB') escape sequences.
 - BGM/TEXT format: Allow \z{head,value} for Nintendos 0x1a escape sequence.
 - Bug fix for wszst+wbmgt DIFF: Some files only in source #2 were not listed.
 - New command: wszst NORMALIZE: resort and recompress an U8 archive.
 - New option: wszst --norm: Force normalization while copying/converting.
 - Many layout things.

wszst (Wiimms SZS Tool)

wszst is a command driven tool to manipulate SZS files.

Features

  • wszst supports compressed and uncompressed SZS files. Uncompressed files are good for analysis. The format of each source file is detected independent of the file name.
  • wszst LIST lists the content of a SZS files and directories. It lists also extracted file systems to preview the creation process.
  • wszst DIFF compares 2 SZS files on file level. This comparing is independent of the compression and of file order.
  • wszst NORMALIZE read an U8 archive, compressed or not, and write it back in a normalized form.
  • wszst COMPRESS and wszst DECOMPRESS convert SZS files from and to compressed format.
  • wszst CREATE creates SZS files from directory structures. All files, but not files starting with a point, are included.
  • wszst EXTRACT extract the directories and files to the local file system to make access very easy.
  • Extrating a SZS file and creating again results in the same SZS file. Only the order of the files may be changed.
  • wszst CAT allow to access one or more sub files within SZS files. The output of all sources is concatenated and written to stdout.


wszst HELP

The HELP command print the built in help. Type wszst HELP for a general help. Add a command to print command specific help:

wszst HELP

wszst LIST

The command LIST list the sub files of SZS files.

wszst LIST

Example:

# wszst lll pool/ref/szs/Award_G.szs

* Files of pool/ref/szs/Award_G.szs

off/hex siz/hex size/dec  file or directory/
-------------------------------------------------------------------------------
      -       -        -  ./
      -       -        -  ./award/
      -       -        -  ./dpd_pointer/
      -       -        -  ./message/
    120    2580     9600  ./message/Common.bmg
   26a0   18300    99072  ./message/Menu.bmg
  1a9a0    1fc0     8128  ./message/Race.bmg
      -       -        -  ./message_window/
      -       -        -  ./pad_recognize/
      -       -        -  ./winning_run/

wszst DIFF

The command DIFF compares two sources (each SZS, U8 or directory) on file level and report mismatches for each sub file.

wszst DIFF

Example:

# wszst diff pool/ref/szs/Award_G.szs pool/ref/szs/MenuMulti_G.szs
* Only in source #1: ./award/
* Only in source #2: ./bg/
* Only in source #2: ./button/
* Only in source #2: ./control/
* Only in source #2: ./globe/
* Only in source #1: ./message/Race.bmg
* Only in source #2: ./model/
Content differ: pool/ref/szs/Award_G.szs : pool/ref/szs/MenuMulti_G.szs

wszst NORMALIZE

The command NORMALIZE read an U8 archive, compressed or not, and write it back in a normalized form:

  • The files are sorted in ASCII order.
  • Unused data holes are removed.
  • '.' becomes the root directory for all others.
  • The maximum available compression is used.

wszst NORMALIZE

wszst COMPRESS

The command COMPRESS compresses a binary file and creates SZS file.

wszst COMPRESS

wszst DECOMPRESS

The command DECOMPRESS decompresses a SZS file and stores the decompressed data into file.

wszst DECOMPRESS

wszst CREATE

The command CREATE creates a SZS file by adding all files of the directory structure.

wszst CREATE

wszst EXTRACT

The command EXTRACT extracts the sub file of source files and copy the files into a directory structure.

wszst EXTRACT

wszst CAT

The command CAT one or more named sub files of SZS and U8 archives. The output of all sources is concatenated and written to stdout.

wszst CAT

wbmgt (Wiimms BMG Tool)

wbmgt is a command driven tool to manipulate BMG files.

Features

  • wbmgt supports raw and text BMG files. Raw BMG is the file format needed by Nintendos software. Text BMG is a user readable and editable format. In each text export an little syntax documentation is included.
  • When reading a file, wbmgt detect the format of each source file independent of the file name. This make the usage of raw and text BMG files interchangable.
  • wbmgt may read BMG files directly from SZS files. Just enter the path and assume that the SZS file is a directory.
  • wbmgt LIST lists the contained strings for a fast preview.
  • wbmgt DIFF compares the strings of 2 BMG files and reports the differences.
  • wbmgt PATCH patches each source by using a list of patch files. There are several patching modes: REPLACE, INSERT, OVERWRITE, DELETE, MASK, EQUAL and NONEQUAL. Like every source, raw and text BMG files are accepted as patch.
  • wbmgt DECODE converts each source file into text mode. Patching while decoding is possible.
  • wbmgt ENCODE converts each source file into raw mode. Patching while encoding is possible.
  • Decoding a raw BMG file and encoding again results in the same BMG file.

Tests

I have made several tests with the following files:

  • All files of Scene/UI/.
  • All files of Race/Course/, some of the tracks have been replaced by custom tracks before.

The tests:

  • Decompress and compress and decompress again SZS files and compare the results.
  • Extract all files, create SZS and extract again. The both extracted file system are equal.
  • Extract some of the files with SZS Explorer and compare them to my extracted files.
  • Extract all files and create a new SZS files. Then create a new ISO image with the new SZS files and play it.
  • Read BMG file, convert to internal representation and write a new BMG. Then compare the new BMG with the original one.
  • Convert all BMG files into the text format and back again. Then compare the new BMG with the original one.

All tests are run without errors.

Links