Difference between revisions of "Wiimms SZS Tools"

From Custom Mario Kart
Jump to navigation Jump to search
(SZS Tools v1.55b released)
(Wiimms SZS Tools v2.13a released)
(47 intermediate revisions by 3 users not shown)
Line 7: Line 7:
 
|-
 
|-
 
! Operating Systems:
 
! Operating Systems:
| Linux (i386,x86_64),<br/>Mac (i386,x86_64),<br/>Windows (cygwin).
+
| Linux (i386,x86_64),<br/>Windows (Cygwin).
 
|-
 
|-
 
! Software Type:
 
! Software Type:
Line 19: Line 19:
 
|-
 
|-
 
! File Formats:
 
! File Formats:
| [[BMG]], [[BTI]], [[KCL]], [[KMP]], [[MDL0]], [[PAT0]], [[TPL]], [[TEX0]], [[PNG]], [[OBJ]], [[StaticR.rel]], [[main.dol]], CT-CODE.
+
| [[BMG]], [[BTI]], [[KCL]], [[KMG]], [[KMP]], [[MDL0]], [[PAT0]], [[TPL]], [[TEX0]], [[PNG]], [[OBJ]], [[StaticR.rel]], [[main.dol]], [[CT-CODE]], [[LE-CODE]], [[LEX]], [[ObjFlow.bin|ObjFlow]], [[GeoHitTableItem.bin|GeoHit]], [[ItemSlot.bin|ItemSlot]], [[minigame.kmg|minigame]].
 
|-
 
|-
 
! Image formats:
 
! Image formats:
Line 25: Line 25:
 
|-
 
|-
 
! Current Version:
 
! Current Version:
| v1.55b, 2017-11-19
+
| v2.13a, 2020-02-15
 
|-
 
|-
 
! Website:
 
! Website:
Line 44: Line 44:
 
* '''wstrt''' : Wiimms StaticR Tool (handles [[StaticR.rel]] and main.dol files of [[Mario Kart Wii]]).
 
* '''wstrt''' : Wiimms StaticR Tool (handles [[StaticR.rel]] and main.dol files of [[Mario Kart Wii]]).
 
* '''wctct''' : Wiimms CT-CODE Tool (handles CT(GP)-CODE to allow >32 tracks).
 
* '''wctct''' : Wiimms CT-CODE Tool (handles CT(GP)-CODE to allow >32 tracks).
 +
* '''wlect''' : Wiimms LE-CODE Tool (handles the [[LE-CODE]] extension by [[User:Leseratte|Leseratte]])
 
One main goal is the fully automated [[Wiimms SZS Tools/How To#Replace Track Names|track name replacement]]. The tools together can do this. Another goal is to extract all sub files recursively and transforming they into user friendly file formats (like text of png files) and support recreation of the original source files.
 
One main goal is the fully automated [[Wiimms SZS Tools/How To#Replace Track Names|track name replacement]]. The tools together can do this. Another goal is to extract all sub files recursively and transforming they into user friendly file formats (like text of png files) and support recreation of the original source files.
  
'''There is also an little [[Wiimms SZS Tools/How To|How To]].'''
+
'''There is also a little [[Wiimms SZS Tools/How To|How To]].'''
  
 
== Main Features ==
 
== Main Features ==
  
* The tools are ''command line tools''. This means, that there is no GUI support. The main advantage is, that you can place commands into scripts (batch files) for fully automated execution &rarr; [https://szs.wiimm.se/info/cli More about ''command line''].
+
* The tools are ''command line tools''. This means, that there is no GUI support. The main advantage is, that you can place commands into scripts (batch files) for fully automated execution &rarr; [https://szs.wiimm.de/info/cli More about ''command line''].
 
* '''wszst''' is the main [[SZS]] tool. It has a fast [[Yaz0]] compressor with the best compression rates.
 
* '''wszst''' is the main [[SZS]] tool. It has a fast [[Yaz0]] compressor with the best compression rates.
 
* The '''wszst''' command '''CHECK''' can help you to find bugs in your track.
 
* The '''wszst''' command '''CHECK''' can help you to find bugs in your track.
Line 67: Line 68:
 
** It can automatically create mipmaps, or it can use user defined mipmaps and create only missed mipmaps automatically.
 
** It can automatically create mipmaps, or it can use user defined mipmaps and create only missed mipmaps automatically.
 
** It can patch images with other images.
 
** It can patch images with other images.
 +
 +
== Last Features ==
 +
 +
=== <span id=bmg>BMG</span> ===
 +
 +
As of v2.08a all 4 known [[BMG]] encodigs (CP1252, UTF-16, Shift-JIS and UTF-8) are supported. The attribute size has been increased from 16 to 36 bytes to support some more games.
 +
 +
=== <span id=lecode>LE-CODE</span> ===
 +
 +
[[LE-CODE]] is a new MKW extension and works like [[CT-CODE]]. To support it, the new tool '''<tt>wlect</tt>''' was created. The tool and the complete LE-CODE support is experimental. No documentation is available, support will be denied for the moment, and there currently is no way for normal end users to create a [[LE-CODE]] distribution.
 +
 +
=== <span id=mipmap>Mipmaps</span> ===
 +
 +
As of v1.65a, the [[mipmap]] creation and the related options changed. Follow the links to SZS Docu for more details:
 +
* [https://szs.wiimm.de/info/mipmap.html General mipmap discussion]
 +
** [https://szs.wiimm.de/info/mipmap.html#opt Options to control mipmaps]
 +
** [https://szs.wiimm.de/info/mipmap.html#resize Resize algorithm (used to create mipmaps)]
  
 
== History and Download ==
 
== History and Download ==
  
[[Wiimm]] started the development of his SZS Tools in March 2011. The first goal was to automate the insertions of the track names for [[Wiimms Mario Kart Fun]] distributions. Therefore 2 tools have been needed: A [[SZS]] extractor and creator and a [[BMG]] manipulator. The 2 tools '''wszst''' and '''wbmgt''' were ready in April 2011. After this, [[Wiimm]] implemented an Image-Tool ('''wimgt'''),  an [[StaticR.rel]] tool ('''wstrt'''), a [[KMP]] tool ('''wkmpt''') and a [[KCL]] tool ('''wkclt'''). All tools are command line tools to support scripts and fully automated operations.
+
[[Wiimm]] started the development of his SZS Tools in March 2011. The first goal was to automate the insertions of the track names for [[Wiimms Mario Kart Fun]] distributions. Therefore 2 tools have been needed: A [[SZS]] extractor and creator and a [[BMG]] manipulator. The 2 tools '''wszst''' and '''wbmgt''' were ready in April 2011. After this, [[Wiimm]] implemented an Image-Tool ('''wimgt'''),  an [[StaticR.rel]] tool ('''wstrt'''), a [[KMP]] tool ('''wkmpt''') and a [[KCL]] tool ('''wkclt'''). Other tools were added later. The last added tool is '''wlect''' to manage [[LE-CODE]] and [[LEX]] files. All tools are command line tools to support scripts and fully automated operations.
  
 
=== Latest Change Log ===
 
=== Latest Change Log ===
  
Since v1.45, the tools wctct and wstrt can use external CT-CODE files, e.g that one created by [https://github.com/Wiimm/wii-ct-code github.com/Wiimm/wii-ct-code]. The internal CT-CODE is now updated to the most current version of the GitHub archive.
+
<pre>
 +
szs v2.13a r8146 - 2020-02-15
  
Since v1.45, the KMP can create fully automated respawn points. With v1.46 flags for KMP sections ENPT, ITPT and JGPT are supported to exclude single points from automatic calculations (fall-down, snap, respawn).
+
- XPF: Added support for negation flag of references to definition objects.
  
<pre>
+
- New keywords for option --kmp:
szs v1.55b r7437 - 2017-11-19
+
    - RM-LECODE: Remove all objects from section GOBJ, that are only relevant
 +
      for LE-CODE. The result is a minimized section for standard code.
 +
    - PURGE-GOBJ: Remove all objects from section GOBJ that are neither a
 +
      valid object nor a definiton object for LE-CODE.
  
  - wszst DISTRIBUTION: New parameter: PREDECESSOR
+
  - wszst ANALYSE + Check KMP: Analysis of KTPT, ENPT and ITPT to find values
 +
  outside range +-131071 where items can not be used online. Additionally a
 +
  recommendation for using option --shift and/or LEX section SET1 is printed.
  
- wszst AUTOADD: With option -v, the command will list invalid files too.
 
  
 +
szs v2.12b r8135 - 2020-02-09
  
szs v1.55a r7429 - 2017-11-11
+
- Bug fixes for LEX parser:
 +
    - Section TEST: Command @ENGINE wasn't implemented.
 +
    - Symbols LEX$OO_AUTO and LEX$GM_AUTO were not defined.
  
  - New command: wszst DISTRIBUTION: The aim is to create a text file that
+
  - New option: --lex-purge: Delete LEX sections without any impact. Delete
  describes a custom track distribution. The owner have to edit the created
+
   sub-file course.lex if it no longer contains a section.
   file and send it to Wiimm. Wiimm add the information to the custom track
 
  portal https://ct.wiimm.de/ with a complete track list.
 
  
 +
- Tools wstrt option --clean-dol: HTTPS modifications are now also reset.
  
szs v1.54a r7393 - 2017-08-14
 
  
- Some KCL flags renamed.
+
szs v2.12a r8129 - 2020-02-07
  
  - wkmpt draw --draw AREA: Areas of KMP are drawn as yellow arrows and as
+
  - Parser: Bug fix for shift-right operator if shifted value is an integer.
  transparent yellow cuboids.
 
  
  - Support for RKC (competition) files added:
+
  - If using --force or --kmp=force, the KMP scanner ignores the size stored in
  RKC files are handled like other archives. Tool wszst is able to detect,
+
   the file header and uses the file size instead.
   list, extract or create a RKC file, also recursive. Direct creation and
 
  optimation of the included SZS is not implemented yet.
 
  
  - New keyword for option --kmp: MAX-LAPS: Keyword MAX-LAPS has only impact,
+
  - New command: wkmpt diff: Compare the KMP of two or more sources (KMP,
   if one of 1LAP,...,9LAPS is set too. Then, the number of laps is only
+
   KMP-text or SZS). Each KMP will be converted into the internal format,
   decreased to the specified number, but never increased.
+
  fixed and patched controlled by options. The known sections are compared,
 +
   some of them sorted before.
  
  - New keyword for option --kmp: RM-EMPTY: After creating a new KMP, all empty
+
  - New option to support wkmpt diff: --diff=list: List of KMP sections names
   KMP sections are removed to make the KMP as small as possible. Use this
+
   that should be compared. Additional keywords possible.
  option only for compettitions!
 
  
  - New command: YAZDUMP: Prints an info dump about the YAZ0/YAZ1 compression
+
  - New option to support wkmpt diff: --epsilon=float: When comparing floating
   details for analysis.
+
   point numbers, 2 numbers are considered identical if their difference is
 +
  less than or equal to epsilon.
  
  - New compression methods: 10 (=ULTRA) and 100-150: They need much more time
+
  - New option: --repair-xpflags: Load a KMP file and use it to repair the
   than BEST (=9) but create smaller SZS files than Nintendo did. It is
+
  settings of the extended presence flags, if they are destroyed by another
  dedicated to competitions with strict size limitations.
+
   KMP tool. --repxpf is a short cut for this option.
  
  - There is a new option: --tiny=level: It tries to minimze SZS and also the
+
  - Option --auto-add was broken for objects, if flags (bits 10-12) were set.
  included KCL and KMP by different technics (rounding values, sorting and
+
   Bit 12 is used by extended presence flags (XPF).
  compression methods 100-150). It is not fully documentd yet.
 
   See https://szs.wiimm.de/info/tiny.html for details.
 
  
  - New keyword for option --kcl: WEAK-WALLS: Set the most significant bit
+
  - Support of random selections of KMP/GOBJ objects:
  (bit 15, 0x8000) of KCL flag of all walls. It stops hard bumping at the
+
    - 6 random scenarios supported.
  edges and has similar effects like lowering the walls.
+
    - New keyword for option --gamemodes: RANDOM.
 +
    - Command wkmpt GAMEMODES can list and command wszst DUPLICATE can create
 +
      up to 1573 variants now.
 +
    - Added a LEX/TEST parameter to force a random mode for testing.
  
  - Windows only: Cygwin update to v2.8.1, 2017-07-03.
+
  - New LEX section SET1 for various settings. The first parameter defines 3
 +
  factors to increase the range for items outside -131071 and +131071.
  
  - Apple Mac is no longer supported because I no longer have access to a Mac.
+
  - New LEX section HIPT: Define places, where the position tracker is hidden.
 
</pre>
 
</pre>
  
Line 216: Line 240:
 
** [[Automated height correction]]
 
** [[Automated height correction]]
  
{{Custom Track Tutorial}}
 
 
{{User-Wiimm-Link}}
 
{{User-Wiimm-Link}}
 
[[category:Software]]
 
[[category:Software]]
 +
[[category:Wiimms SZS Tools]]
 +
[[category:LE-CODE]]

Revision as of 09:55, 15 February 2020

Wiimms SZS Tools
File:Wiimms-SZS-Tools.png
Author: Wiimm
Operating Systems: Linux (i386,x86_64),
Windows (Cygwin).
Software Type: File manager, decoding and encoding toolset.
Compression Formats: Yaz0, Yaz1, WBZ.
Archive Formats: SZS, U8, WU8, PACK, RARC, BRRES, BREFF, BREFT.
File Formats: BMG, BTI, KCL, KMG, KMP, MDL0, PAT0, TPL, TEX0, PNG, OBJ, StaticR.rel, main.dol, CT-CODE, LE-CODE, LEX, ObjFlow, GeoHit, ItemSlot, minigame.
Image formats: I4, I8, IA4, IA8, RGB565, RGB5A3, RGBA32, C4, C8, C14X2, CMPR.
Current Version: v2.13a, 2020-02-15
Website: szs.wiimm.de

Wiimms SZS Tools has a spelling error, you should see but as a brand name.

Introduction

Wiimms SZS Tools is a set of different tools to manipulate SZS, U8, RARC, BRRES, BREFF and BREFT archives and BMG, BTI, KCL, KMP, MDL0, TPL, TEX0 and StaticR.rel files. All tools are command line tools and run without any interaction. The tools are available for Linux, Mac and Windows. The main goal is to run them in batch files and scripts to automate recurring jobs. At the moment there are 8 tools:

  • wszst : Wiimms SZS Tool (handles SZS, U8, RARC, BRRES, BREFF and BREFT archives)
  • wbmgt : Wiimms BMG Tool (convert and patch BMG files in binary and text format).
  • wimgt : Wiimms Image Tool (convert and patch image files of Mario Kart Wii).
  • wkclt : Wiimms KCL Tool (convert raw KCL files from/into obj format).
  • wkmpt : Wiimms KMP Tool (handles raw and text KMP files, read text files like a compiler).
  • wpatt : Wiimms PAT Tool (handles raw and text PAT0 files, read text files like a compiler).
  • wstrt : Wiimms StaticR Tool (handles StaticR.rel and main.dol files of Mario Kart Wii).
  • wctct : Wiimms CT-CODE Tool (handles CT(GP)-CODE to allow >32 tracks).
  • wlect : Wiimms LE-CODE Tool (handles the LE-CODE extension by Leseratte)

One main goal is the fully automated track name replacement. The tools together can do this. Another goal is to extract all sub files recursively and transforming they into user friendly file formats (like text of png files) and support recreation of the original source files.

There is also a little How To.

Main Features

  • The tools are command line tools. This means, that there is no GUI support. The main advantage is, that you can place commands into scripts (batch files) for fully automated execution → More about command line.
  • wszst is the main SZS tool. It has a fast Yaz0 compressor with the best compression rates.
  • The wszst command CHECK can help you to find bugs in your track.
  • The new wszst command SLOTS makes a track proposal for working and non working slots.
  • The KCL tool wkclt can create KCL files from scratch (they become as small as possible) by reading an OBJ file. It can also export an existing KCL to an OBJ file with colors to make a KCL visible.
  • The KMP tool wkmpt can decode and compile (yes, it's a real compiler with loops and other control structures) KMP files.
    • While compiling, it can overide some parameters for debugging or optimization.
    • The KMP compiler can use the KCL to find optimal vertical positions of objects and route points. This is good for visual debugging.
  • The wkmpt command DRAW creates an OBJ file with KCL and add route positions and other objects of the KMP.
  • The wszst command PATCH can transform whole tracks inkl. KCL, KMP, MODEL, MINIMAP and VRCORN. Transforming means any combinations of scaling, rotating and shifting with different factors for each axis.
  • The wszst command MINIMAP can center the minimap automatically. You can use this command with SZS, U8, BRRES and MDL0 files.
  • The image tool wimgt can convert images of all formats:
    • It can convert the file formats TPL, TEX0, BTI, BREFT and PNG to any other.
    • It can convert the Image Formats I4, I8, IA4, IA8, RGB565, RGB5A3, RGBA32, C4, C8, C14X2 and CMPR to any other.
    • It supports the palette formats IA8, RGB565 and RGB5A3.
    • It can automatically create mipmaps, or it can use user defined mipmaps and create only missed mipmaps automatically.
    • It can patch images with other images.

Last Features

BMG

As of v2.08a all 4 known BMG encodigs (CP1252, UTF-16, Shift-JIS and UTF-8) are supported. The attribute size has been increased from 16 to 36 bytes to support some more games.

LE-CODE

LE-CODE is a new MKW extension and works like CT-CODE. To support it, the new tool wlect was created. The tool and the complete LE-CODE support is experimental. No documentation is available, support will be denied for the moment, and there currently is no way for normal end users to create a LE-CODE distribution.

Mipmaps

As of v1.65a, the mipmap creation and the related options changed. Follow the links to SZS Docu for more details:

History and Download

Wiimm started the development of his SZS Tools in March 2011. The first goal was to automate the insertions of the track names for Wiimms Mario Kart Fun distributions. Therefore 2 tools have been needed: A SZS extractor and creator and a BMG manipulator. The 2 tools wszst and wbmgt were ready in April 2011. After this, Wiimm implemented an Image-Tool (wimgt), an StaticR.rel tool (wstrt), a KMP tool (wkmpt) and a KCL tool (wkclt). Other tools were added later. The last added tool is wlect to manage LE-CODE and LEX files. All tools are command line tools to support scripts and fully automated operations.

Latest Change Log

szs v2.13a r8146 - 2020-02-15

 - XPF: Added support for negation flag of references to definition objects.

 - New keywords for option --kmp:
    - RM-LECODE: Remove all objects from section GOBJ, that are only relevant
      for LE-CODE. The result is a minimized section for standard code.
    - PURGE-GOBJ: Remove all objects from section GOBJ that are neither a
      valid object nor a definiton object for LE-CODE.

 - wszst ANALYSE + Check KMP: Analysis of KTPT, ENPT and ITPT to find values
   outside range +-131071 where items can not be used online. Additionally a
   recommendation for using option --shift and/or LEX section SET1 is printed.


szs v2.12b r8135 - 2020-02-09

 - Bug fixes for LEX parser:
    - Section TEST: Command @ENGINE wasn't implemented.
    - Symbols LEX$OO_AUTO and LEX$GM_AUTO were not defined.

 - New option: --lex-purge: Delete LEX sections without any impact. Delete
   sub-file course.lex if it no longer contains a section.

 - Tools wstrt option --clean-dol: HTTPS modifications are now also reset.


szs v2.12a r8129 - 2020-02-07

 - Parser: Bug fix for shift-right operator if shifted value is an integer.

 - If using --force or --kmp=force, the KMP scanner ignores the size stored in
   the file header and uses the file size instead.

 - New command: wkmpt diff: Compare the KMP of two or more sources (KMP,
   KMP-text or SZS). Each KMP will be converted into the internal format,
   fixed and patched controlled by options. The known sections are compared,
   some of them sorted before.

 - New option to support wkmpt diff: --diff=list: List of KMP sections names
   that should be compared. Additional keywords possible.

 - New option to support wkmpt diff: --epsilon=float: When comparing floating
   point numbers, 2 numbers are considered identical if their difference is
   less than or equal to epsilon.

 - New option: --repair-xpflags: Load a KMP file and use it to repair the
   settings of the extended presence flags, if they are destroyed by another
   KMP tool. --repxpf is a short cut for this option.

 - Option --auto-add was broken for objects, if flags (bits 10-12) were set.
   Bit 12 is used by extended presence flags (XPF).

 - Support of random selections of KMP/GOBJ objects:
    - 6 random scenarios supported.
    - New keyword for option --gamemodes: RANDOM.
    - Command wkmpt GAMEMODES can list and command wszst DUPLICATE can create
      up to 1573 variants now.
    - Added a LEX/TEST parameter to force a random mode for testing.

 - New LEX section SET1 for various settings. The first parameter defines 3
   factors to increase the range for items outside -131071 and +131071.

 - New LEX section HIPT: Define places, where the position tracker is hidden.

Links

Tools Overview

wszst (Wiimms SZS Tool)

wszst is a command driven tool to manipulate archives:

  • Supported archive formats: SZS, U8, RARC, BRRES, BREFF and BREFT.
  • It can extract all subfiles of any archive.
  • While extracting some file formats can be decoded (converted to text or PNG files).
  • It can create all (but not RARC archives) while readind and scanning a directory structure. Decoded files are encoded automatically.

Tool wszst: Features, commands and options

wbmgt (Wiimms BMG Tool)

wbmgt is a command driven tool to manipulate BMG (message) files.
Tool wbmgt: Features, commands and options

wkclt (Wiimms KCL Tool)

wkclt is a command driven tool to convert KCL files into/from binary KCL from/into Wavefront OBJ files.
Tool wkclt: Features, commands and options
General parser syntax and semantics
KCL export and import guide

wkmpt (Wiimms KMP Tool)

wkmpt is a command driven tool to convert KMP files into/from binary from/into text files. A special compiler is used to allow variables, expressions, if..then..else, loops, macros and user defined functions while reading a KMP text file.
Tool wkmpt: Features, commands and options
General parser syntax and semantics
KMP text syntax and semantics
KMP parser functions
How to edit KMP files

wimgt (Wiimms Image Tool)

wimgt is a command driven tool to convert and aptch graphic images:

  • wimgt supports the file formats BREFT, BTI, TEX0, TPL and PNG as export and import type.
  • wimgt supports the internal image formats I4, I8, IA4, IA8, RGB565, RGB5A3, RGBA32, C4, C8, C14X2 and CMPR (and also different PNG formats).

Tool wimgt: Features, commands and options

wstrt (Wiimms StaticR Tool)

wstrt is a command driven tool to manipulate StaticR files of Mario Kart Wii.
Tool wstrt: Features, commands and options

Links