Difference between revisions of "Wiimms SZS Tools"

From Custom Mario Kart
Jump to navigation Jump to search
(Wiimms SZS Tools v1.54a released)
(Wiimms SZS Tools v1.58a released)
(10 intermediate revisions by the same user not shown)
Line 25: Line 25:
 
|-
 
|-
 
! Current Version:
 
! Current Version:
| v1.54a, 2017-08-14
+
| v1.58a, 2018-04-20
 
|-
 
|-
 
! Website:
 
! Website:
Line 67: Line 67:
 
** It can automatically create mipmaps, or it can use user defined mipmaps and create only missed mipmaps automatically.
 
** It can automatically create mipmaps, or it can use user defined mipmaps and create only missed mipmaps automatically.
 
** It can patch images with other images.
 
** It can patch images with other images.
 +
 +
=== <span id=small>Ignoring small triangles</span> ===
 +
 +
Sometimes, the imported file contains very small triangles with an area smaller than 1 unit<sup>2</sup>. They were accidentally inserted into the model by the user. Because of the littleness, the user will not see them in the model view of its [[3D Tool]]. However, machines like the [[KCL]] importer of these tools see them and insert them into the KCL &mdash; this is not a bug of the tools. The Wii has (mathematical) problems to use them in a correct way with the result of KCL glitches like mini walls on the road.
 +
 +
Since v1.58a there exist 2 new options: '''--tri-area=EXPR''' and '''--tri-height=EXPR'''[https://szs.wiimm.de/info/kcl-guide.html#tri-min] to filter out too small triangles.
 +
 +
Option [https://szs.wiimm.de/opt/tri-area --tri-area=EXPR] defines the minimal area size of KCL triangles. The intention is to '''ignore triangles that are generally to small'''. EXPR is a floating point number or expression. Triangles are invalidated if their area size is smaller than EXPR. Values between 0.01 and 4.0 are recommended. The careful value 1.0 is used as default. Value 0 disables this filter functionality.
 +
 +
Option [https://szs.wiimm.de/opt/tri-height --tri-height=EXPR] defines the minimal height of KCL triangles. The intention is to '''ignore deformed triangles (very slim, but long)'''. EXPR is a floating point number or expression. Triangles are invalidated if at least 1 of the 3 heights is smaller than EXPR. Values between 0.01 and 2.0 are recommended. The careful value 0.1 is used as default. Value 0 disables this filter functionality.
 +
 +
The tests for both options are executed after reading files, after transformations, and after calculating normals and lengths.
  
 
== History and Download ==
 
== History and Download ==
Line 79: Line 91:
  
 
<pre>
 
<pre>
szs v1.54a r7393 - 2017-08-14
+
szs v1.58a r7499 - 2018-04-20
 
 
- Some KCL flags renamed.
 
 
 
- wkmpt draw --draw AREA: Areas of KMP are drawn as yellow arrows and as
 
  transparent yellow cuboids.
 
 
 
- Support for RKC (competition) files added:
 
  RKC files are handled like other archives. Tool wszst is able to detect,
 
  list, extract or create a RKC file, also recursive. Direct creation and
 
  optimation of the included SZS is not implemented yet.
 
  
  - New keyword for option --kmp: MAX-LAPS: Keyword MAX-LAPS has only impact,
+
  - Option --kcl-min=EXPR: This option is deprecated, but still supported. It
   if one of 1LAP,...,9LAPS is set too. Then, the number of laps is only
+
   is an alternative name for --tri-height=EXPR now.
  decreased to the specified number, but never increased.
 
  
  - New keyword for option --kmp: RM-EMPTY: After creating a new KMP, all empty
+
  - New option --tri-area=EXPR: Define the minimal area size of KCL triangles.
   KMP sections are removed to make the KMP as small as possible. Use this
+
  The intention is to ignore triangles that are generally to small. EXPR is
  option only for compettitions!
+
  a floating point number or expression. Triangles are invalidated if their
 +
  area size is smaller than EXPR. The test is executed after reading files,
 +
   after transformations, and after calculating normals and lengths. Values
 +
  between 0.01 and 4.0 are recommended. The careful value 1.0 is used as
 +
  default. Value 0 disables this filter functionality.
  
  - New command: YAZDUMP: Prints an info dump about the YAZ0/YAZ1 compression
+
  - New option --tri-height=EXPR: Define the minimal height of KCL triangles.
   details for analysis.
+
  The intention is to ignore deformed triangles (very slim, but long). EXPR
 +
  is a floating point number or expression. Triangles are invalidated if at
 +
  least 1 of the 3 heights is smaller than EXPR. The test is executed after
 +
  reading files, after transformations, and after calculating normals and
 +
  lengths. Values between 0.01 and 2.0 are recommended. The careful value 0.1
 +
   is used as default. Value 0 disables this filter functionality.
  
  - New compression methods: 10 (=ULTRA) and 100-150: They need much more time
+
  - You can proof the impact of the options above by command wkclt TEST.
  than BEST (=9) but create smaller SZS files than Nintendo did. It is
 
  dedicated to competitions with strict size limitations.
 
  
  - There is a new option: --tiny=level: It tries to minimze SZS and also the
+
  - Command wkclt DUMP has been expanded and shows metrics like side length,
   included KCL and KMP by different technics (rounding values, sorting and
+
   area size and heights for each triangle.
  compression methods 100-150). It is not fully documentd yet.
 
  See https://szs.wiimm.de/info/tiny.html for details.
 
  
  - New keyword for option --kcl: WEAK-WALLS: Set the most significant bit
+
  - New command: wkclt LIST: Print a list of all triangles with metrics in
   (bit 15, 0x8000) of KCL flag of all walls. It stops hard bumping at the
+
   human and machine readable text format. It is like command DUMP, but
   edges and has similar effects like lowering the walls.
+
   reduced to the list of triangles.
  
  - Windows only: Cygwin update to v2.8.1, 2017-07-03.
+
  - New implementation for OBJ groups without assigned KCL flag. They will be
 +
  inserted into the OBJ export now.
  
- Apple Mac is no longer supported because I no longer have access to a Mac.
 
  
 +
szs v1.57a r7484 - 2018-04-11
  
szs v1.53a r7308 - 2016-12-09
+
- Windows version only: Update to Cygwin v2.10.0.
  
  - Bug fix: Occasional a "Segmentation fault" occured, if reading an empty file.
+
  - New option for tools wkclt, wszst and wkmpt: --kcl-min=FLOAT: Define a
  - Option --draw supports the new keyword DECORATION: Draw unsolid objects
+
  minimal `length` of KCL triangles. The length is the heights of the
   without any route as a magenta cuboid.
+
  reference side of the triangle. Smaller triangles are invalidated and
 +
  ignored or removed. The default value is 0.1 and good values are between
 +
  0.01 and 2.0. Until v1.56a the minimal length was hard coded as 1e-6
 +
  (1/million) to avoid mathematical overflows while calculating positions
 +
  by normals. The larger value reduces the number of KCL glitches
 +
   dramatically for some tracks with many unusual triangles.
  
 +
- New option: --wc24: If set, options --wiimmfi and --domain will exclude 2
 +
  strings of StaticR.rel from the URL patch to enable WC24 support.
  
szs v1.52a r7164 - 2016-06-29
 
  
- Bug fix for wkclt: Since v1.49a, the transformation was executed twice,
+
szs v1.56a r7461 - 2017-12-03
  but only for commands DECODE and ENCODE. Command COPY works correct and
 
  transforms only once.
 
  
  - wstrt: Wiimmfi patching excludes now 3 patch positions for competitions.
+
  - wszst DISTRIBUTION: Track scanning improved.
  It's now the job of the competition patcher to patch these 3 URLs.
 
  
  - wkclt CFF: If creating a new flag file, the new function i(type,z,index)
+
  - Option --cache is a global option now. It is used to speedup every YAZ0
   is defined as support for fall boundaries (idea by Wexos).
+
  compression, but only for standard compression level 9 (BEST). A hash table
 +
   is used now for optimization.
  
  - New tool: wmdlt: The tool wmdlt exists since v0.26a (2012), but was always
+
  - New option: --cname: Defines a cache file name for a single file.
  hidden, because it was useless. But now I started with MDL analysis and
 
  plan to modifiy MDL data.
 
    - wmdlt CAT/DECODE: Print a MDL file as text. It is incomplete, but can be
 
      used for analysing.
 
    - wmdlt STRINGS: Print all strings of a MDL.
 
  
  - For slot 42 (moonview highway) the content of the 12 needed materials
+
  - New Command: wszst SCANCACHE: Scan the cache directory for SZS files and
   is checked.
+
   re-build the hash table.
 
</pre>
 
</pre>
  

Revision as of 14:28, 20 April 2018

Wiimms SZS Tools
File:Wiimms-SZS-Tools.png
Author: Wiimm
Operating Systems: Linux (i386,x86_64),
Mac (i386,x86_64),
Windows (cygwin).
Software Type: File manager, decoding and encoding toolset.
Compression Formats: Yaz0, Yaz1, WBZ.
Archive Formats: SZS, U8, WU8, PACK, RARC, BRRES, BREFF, BREFT.
File Formats: BMG, BTI, KCL, KMP, MDL0, PAT0, TPL, TEX0, PNG, OBJ, StaticR.rel, main.dol, CT-CODE.
Image formats: I4, I8, IA4, IA8, RGB565, RGB5A3, RGBA32, C4, C8, C14X2, CMPR.
Current Version: v1.58a, 2018-04-20
Website: szs.wiimm.de

Wiimms SZS Tools has a spelling error, you should see but as a brand name.

Introduction

Wiimms SZS Tools is a set of different tools to manipulate SZS, U8, RARC, BRRES, BREFF and BREFT archives and BMG, BTI, KCL, KMP, MDL0, TPL, TEX0 and StaticR.rel files. All tools are command line tools and run without any interaction. The tools are available for Linux, Mac and Windows. The main goal is to run them in batch files and scripts to automate recurring jobs. At the moment there are 8 tools:

  • wszst : Wiimms SZS Tool (handles SZS, U8, RARC, BRRES, BREFF and BREFT archives)
  • wbmgt : Wiimms BMG Tool (convert and patch BMG files in binary and text format).
  • wimgt : Wiimms Image Tool (convert and patch image files of Mario Kart Wii).
  • wkclt : Wiimms KCL Tool (convert raw KCL files from/into obj format).
  • wkmpt : Wiimms KMP Tool (handles raw and text KMP files, read text files like a compiler).
  • wpatt : Wiimms PAT Tool (handles raw and text PAT0 files, read text files like a compiler).
  • wstrt : Wiimms StaticR Tool (handles StaticR.rel and main.dol files of Mario Kart Wii).
  • wctct : Wiimms CT-CODE Tool (handles CT(GP)-CODE to allow >32 tracks).

One main goal is the fully automated track name replacement. The tools together can do this. Another goal is to extract all sub files recursively and transforming they into user friendly file formats (like text of png files) and support recreation of the original source files.

There is also an little How To.

Main Features

  • The tools are command line tools. This means, that there is no GUI support. The main advantage is, that you can place commands into scripts (batch files) for fully automated execution → More about command line.
  • wszst is the main SZS tool. It has a fast Yaz0 compressor with the best compression rates.
  • The wszst command CHECK can help you to find bugs in your track.
  • The new wszst command SLOTS makes a track proposal for working and non working slots.
  • The KCL tool wkclt can create KCL files from scratch (they become as small as possible) by reading an OBJ file. It can also export an existing KCL to an OBJ file with colors to make a KCL visible.
  • The KMP tool wkmpt can decode and compile (yes, it's a real compiler with loops and other control structures) KMP files.
    • While compiling, it can overide some parameters for debugging or optimization.
    • The KMP compiler can use the KCL to find optimal vertical positions of objects and route points. This is good for visual debugging.
  • The wkmpt command DRAW creates an OBJ file with KCL and add route positions and other objects of the KMP.
  • The wszst command PATCH can transform whole tracks inkl. KCL, KMP, MODEL, MINIMAP and VRCORN. Transforming means any combinations of scaling, rotating and shifting with different factors for each axis.
  • The wszst command MINIMAP can center the minimap automatically. You can use this command with SZS, U8, BRRES and MDL0 files.
  • The image tool wimgt can convert images of all formats:
    • It can convert the file formats TPL, TEX0, BTI, BREFT and PNG to any other.
    • It can convert the Image Formats I4, I8, IA4, IA8, RGB565, RGB5A3, RGBA32, C4, C8, C14X2 and CMPR to any other.
    • It supports the palette formats IA8, RGB565 and RGB5A3.
    • It can automatically create mipmaps, or it can use user defined mipmaps and create only missed mipmaps automatically.
    • It can patch images with other images.

Ignoring small triangles

Sometimes, the imported file contains very small triangles with an area smaller than 1 unit2. They were accidentally inserted into the model by the user. Because of the littleness, the user will not see them in the model view of its 3D Tool. However, machines like the KCL importer of these tools see them and insert them into the KCL — this is not a bug of the tools. The Wii has (mathematical) problems to use them in a correct way with the result of KCL glitches like mini walls on the road.

Since v1.58a there exist 2 new options: --tri-area=EXPR and --tri-height=EXPR[1] to filter out too small triangles.

Option --tri-area=EXPR defines the minimal area size of KCL triangles. The intention is to ignore triangles that are generally to small. EXPR is a floating point number or expression. Triangles are invalidated if their area size is smaller than EXPR. Values between 0.01 and 4.0 are recommended. The careful value 1.0 is used as default. Value 0 disables this filter functionality.

Option --tri-height=EXPR defines the minimal height of KCL triangles. The intention is to ignore deformed triangles (very slim, but long). EXPR is a floating point number or expression. Triangles are invalidated if at least 1 of the 3 heights is smaller than EXPR. Values between 0.01 and 2.0 are recommended. The careful value 0.1 is used as default. Value 0 disables this filter functionality.

The tests for both options are executed after reading files, after transformations, and after calculating normals and lengths.

History and Download

Wiimm started the development of his SZS Tools in March 2011. The first goal was to automate the insertions of the track names for Wiimms Mario Kart Fun distributions. Therefore 2 tools have been needed: A SZS extractor and creator and a BMG manipulator. The 2 tools wszst and wbmgt were ready in April 2011. After this, Wiimm implemented an Image-Tool (wimgt), an StaticR.rel tool (wstrt), a KMP tool (wkmpt) and a KCL tool (wkclt). All tools are command line tools to support scripts and fully automated operations.

Latest Change Log

Since v1.45, the tools wctct and wstrt can use external CT-CODE files, e.g that one created by github.com/Wiimm/wii-ct-code. The internal CT-CODE is now updated to the most current version of the GitHub archive.

Since v1.45, the KMP can create fully automated respawn points. With v1.46 flags for KMP sections ENPT, ITPT and JGPT are supported to exclude single points from automatic calculations (fall-down, snap, respawn).

szs v1.58a r7499 - 2018-04-20

 - Option --kcl-min=EXPR: This option is deprecated, but still supported. It
   is an alternative name for --tri-height=EXPR now.

 - New option --tri-area=EXPR: Define the minimal area size of KCL triangles.
   The intention is to ignore triangles that are generally to small. EXPR is
   a floating point number or expression. Triangles are invalidated if their
   area size is smaller than EXPR. The test is executed after reading files,
   after transformations, and after calculating normals and lengths. Values
   between 0.01 and 4.0 are recommended. The careful value 1.0 is used as
   default. Value 0 disables this filter functionality.

 - New option --tri-height=EXPR: Define the minimal height of KCL triangles.
   The intention is to ignore deformed triangles (very slim, but long). EXPR
   is a floating point number or expression. Triangles are invalidated if at
   least 1 of the 3 heights is smaller than EXPR. The test is executed after
   reading files, after transformations, and after calculating normals and
   lengths. Values between 0.01 and 2.0 are recommended. The careful value 0.1
   is used as default. Value 0 disables this filter functionality.

 - You can proof the impact of the options above by command wkclt TEST.

 - Command wkclt DUMP has been expanded and shows metrics like side length,
   area size and heights for each triangle.

 - New command: wkclt LIST: Print a list of all triangles with metrics in
   human and machine readable text format. It is like command DUMP, but
   reduced to the list of triangles.

 - New implementation for OBJ groups without assigned KCL flag. They will be
   inserted into the OBJ export now.


szs v1.57a r7484 - 2018-04-11

 - Windows version only: Update to Cygwin v2.10.0.

 - New option for tools wkclt, wszst and wkmpt: --kcl-min=FLOAT: Define a
   minimal `length` of KCL triangles. The length is the heights of the
   reference side of the triangle. Smaller triangles are invalidated and
   ignored or removed. The default value is 0.1 and good values are between
   0.01 and 2.0. Until v1.56a the minimal length was hard coded as 1e-6
   (1/million) to avoid  mathematical overflows while calculating positions
   by normals. The larger value reduces the number of KCL glitches
   dramatically for some tracks with many unusual triangles.

 - New option: --wc24: If set, options --wiimmfi and --domain will exclude 2
   strings of StaticR.rel from the URL patch to enable WC24 support.


szs v1.56a r7461 - 2017-12-03

 - wszst DISTRIBUTION: Track scanning improved.

 - Option --cache is a global option now. It is used to speedup every YAZ0
   compression, but only for standard compression level 9 (BEST). A hash table
   is used now for optimization.

 - New option: --cname: Defines a cache file name for a single file.

 - New Command: wszst SCANCACHE: Scan the cache directory for SZS files and
   re-build the hash table.

Links

Tools Overview

wszst (Wiimms SZS Tool)

wszst is a command driven tool to manipulate archives:

  • Supported archive formats: SZS, U8, RARC, BRRES, BREFF and BREFT.
  • It can extract all subfiles of any archive.
  • While extracting some file formats can be decoded (converted to text or PNG files).
  • It can create all (but not RARC archives) while readind and scanning a directory structure. Decoded files are encoded automatically.

Tool wszst: Features, commands and options

wbmgt (Wiimms BMG Tool)

wbmgt is a command driven tool to manipulate BMG (message) files.
Tool wbmgt: Features, commands and options

wkclt (Wiimms KCL Tool)

wkclt is a command driven tool to convert KCL files into/from binary KCL from/into Wavefront OBJ files.
Tool wkclt: Features, commands and options
General parser syntax and semantics
KCL export and import guide

wkmpt (Wiimms KMP Tool)

wkmpt is a command driven tool to convert KMP files into/from binary from/into text files. A special compiler is used to allow variables, expressions, if..then..else, loops, macros and user defined functions while reading a KMP text file.
Tool wkmpt: Features, commands and options
General parser syntax and semantics
KMP text syntax and semantics
KMP parser functions
How to edit KMP files

wimgt (Wiimms Image Tool)

wimgt is a command driven tool to convert and aptch graphic images:

  • wimgt supports the file formats BREFT, BTI, TEX0, TPL and PNG as export and import type.
  • wimgt supports the internal image formats I4, I8, IA4, IA8, RGB565, RGB5A3, RGBA32, C4, C8, C14X2 and CMPR (and also different PNG formats).

Tool wimgt: Features, commands and options

wstrt (Wiimms StaticR Tool)

wstrt is a command driven tool to manipulate StaticR files of Mario Kart Wii.
Tool wstrt: Features, commands and options

Links