LE-CODE/Distribution Tutorial/LE-DEF

From Custom Mario Kart
Jump to navigation Jump to search
New Definition Format

As of v2.28a.beta of the SZS Tools, a new file format will be supported as an alternative for the old format CT-DEF. It has a simpler syntax and will be added with future LE-CODE extensions. Along with the file format there is also a new command wlect distribution which allows the processing of old and new file formats.

LE-CODE
LE-CODE & related file formats
Extended presence flags
Related Categories

LE-CODE Logo (Circle).png

Introduction

In 2013, Wiimm integrated CT-CODE support into its SZS tools and has continually expanded or improved it. For this purpose he developed the file format CT-DEF, with which the user can carry out the configuration via a text file. Between 2019 and 2022 he extended the CT-DEF format to support LE-CODE. A separate LE-CODE definition format was planned from the start to support battle arenas, among other things. Since 2020, the LPAR file format has been developed to manage all LE-CODE parameters.

In May 2022, Wiimm started to develop a new internal data model for his SZS Tools, which contains all relevant data of a LE-CODE distribution. This development is still ongoing. To manage this new data model, the new command wlect distribution and various new file formats for import and export were developed. One of the new formats is the configuration format LE-DEF (LE-CODE Definition), with which arenas, tracks and cups are defined. This format is to be further developed in step with LE-CODE extensions. Command wlect distribution can write and read this new file format.

The new file formats and commands will be available as of Wiimms SZS Tools v2.28a (Test version v2.28a.beta has been released).

File Format LE-DEF

LE-DEF is the new file format for LE-CODE definitions. Command wlect filetype uses this name to identify the format. The first 8 characters of such file must be »#LE-DEF1« for the identification.

A LE-DEF file is divided into several sections. Section [TRACK-LIST] defines the tracks with all parameters. A minimum LE-DEF file must contain at least this section. Section [CUP-LIST] defines how the arenas and tracks are distributed to the cups.

Command »wlect distribution«

The new command wlect distribution (shorter wlect distrib or even shorter wlect dis: both are well defined abbreviations) manages LE-CODE distributions. It reads any number of source files with different file types, collects the data and creates any number of files with different file types.

wlect dis … is a very powerful command when dealing with distributions. Because of the complexity, the input line offers many possibilities. It is processed step by step from left to right. There are input files that change the internal model. The instructions are used to change the data or to write data to files. The processing options affect both reading and writing.

For more details call wlect dis without arguments. This built-in help is also available as text file.

Command line limits and solutions

For example, let's take the command wlect dis ./path/to/*.szs ... and assume that there are 1000 tracks in the directory ./path/to/. Since the outer shell resolves the *, all file names are written to the command line one after the other. A calculation from the ct.wiimm.de shows that the file names have an average length of 51 characters. There are also 10 characters for the path and 1 character for the separator. This means that the file names of the 1000 tracks alone require 62,000 characters on the command line. In principle, it should even be possible to process up to 4028 tracks. We ignore the additional _d variants for this calculation. For these 4028 tracks, the command line would then have to take up 250,000 characters.

That's quite a lot, but it's not a problem for Linux, which reserves at least 2 MB for the command line. In Cygwin, which is used for Windows, the limit is 32000 characters, and in Windows 10 it is only 8192 characters. Both are not enough for our scenario. Therefore, other solutions must be found, which are an integral part of the SZS tools from v2.28.

The first solution is to move the command line arguments to 1 or more files and then tell the respective tool the filenames. Exactly this is made possible by the new option -@ FILENAME (the space after @ is optional). This can massively reduce the volume of the command line. In a first analysis of the command line is searched for -@.... The option is then replaced by the content of the file. This option can also be used recursively up to level 10. This is done internally, so the only limit is available main memory. This newly composed command line is then examined for options and parameters as usual.

The text files are treated similarly to the command line. Line breaks are used like spaces to separate arguments. Argument such as filenames with spaces must be enclosed in "..." or '...'. Example: 

--brief --load-prefix prefix.txt
"Track number 1.szs"  "Track number 2.szs"
"Track number 3.szs"
./path/to/*.szs

The second solution is that the tool evaluates so-called wildcards itself. In the above example, it would then only have to be prevented that the outer shell evaluates the wildcards. This can be done with "./path/to/*.szs" for example. wlect dis is the first command that supports this internal file search. This evaluation is done after loading text files to the command line by -@ FILENAME.

Debugging: There are 2 commands to verify command lines. Command wlect argtest ... shows how the tool sees the command line. Command wlect expand ... uses all parameters as filenames with wildcards and searches for all matching files. -@ FILENAME has already been evaluated for both commands.

File names used in this article

The same file names appear again and again in the examples so that the reader knows exactly what type of file it is. However, the file names can be chosen arbitrarily. Even file extensions such as .txt are irrelevant, as the content of a file is always examined to identify the file type. Therefore, text files must always begin with the correct identifier (=magic).

List of filenames:

lecode-PAL.bin, lecode-USA.bin, lecode-JAP.bin, lecode-KOR.bin
LE-CODE binary files of each region.
[email protected]
A filename to create 4 LE-CODE binaries, 1 for each region.
le-def.txt
A LE-CODE definition file of type LE-DEF.
lpar.txt
A LE-CODE parameter file of type LPAR.
prefix.txt
A file, that defines the game prefixes. See chapter »Game prefixes« for details.
./path/to/*.szs
A series of arena and track files. See »How to enter ./path/to/…« for a discussion.

Creating a LE-CODE definition file

The syntax of a LE-CODE definition file file is relatively simple. So you can just open a text editor and write down the definition. Don't forget to write the identifier »#LE-DEF1« in the first line.

Alternatively, you can create a template. This is what the command wlect create le-def is for. A description as well as a few sample tracks are inserted here. The command wlect dis le-def=- does the same thing, but without sample tracks. The option --brief (short: -B) can be added to both commands to suppress the description.

Schema of a LE-DEF file: 

#LE-DEF1
[TOOLS]
....
[TRACK-LIST]
....
[CUP-LIST]
....

Since a 2-pass scanner is used, the order of the sections is arbitrary. Section [TOOL] is not needed at the moment. It is also possible to use several sections more than once. Before the first LE-DEF file is scanned the complete track and cup lists of the internal model are cleared. All following LE-DEF files supplement the model. For example, a division into 2 files is possible once for battle arenas and and once for Versus tracks. 

#LE-DEF1 => start of first file
[TRACK-LIST]
....
[CUP-LIST]
....

#LE-DEF1 => start of second file with reversed order => this line is ignored
[CUP-LIST]
....
[TRACK-LIST]
....

Section [TRACK-LIST]

Battle arenas and versus tracks are defined in this section by instruction TRACK. Battle arenas are assigned to slots 32 through 41 first and versus tracks to slots 0 through 31. Additional arenas and tracks will then be assigned from slot 68 (see LE-CODE: Slot usage). Since the specific slots are not a problem, the slots are usually assigned fully automatically. However, if required, slots can also be forcibly assigned by instruction SLOT. Instruction IGNORE-SLOT can disable all SLOT instructions to force an automatic track order.

The two instructions STANDARD-BATTLE-ARENAS and STANDARD-VERSUS-TRACKS are used to fill slots 0 to 41 with the standard tracks. Use them before any other track definition.

A common track definition looks like this: 

IGNORE-SLOT 0
SLOT 123
TRACK varname type property music flags
	laps	2
	speed	1.2
	file    "file_xxx"
	ident   "SHA1 of SZS"
	name    "Track #1"
	xname   "Extended Name #1"
	[key]   "any text"

Analysis:

Instruction IGNORE-SLOT 0
Enable SLOT instructions. This is the default. Available as of SZS Tools v2.30a.
Instruction SLOT 123
Define, that the next defined track is stored at slot 123. An already existing track definition is overridden.
Instructions TRACK, LAPS, SPEED, FILE, IDENT, NAME, XNAME, [key]
These are independent commands (case in-sensitive). TRACK defines a new arena or track and the other commands text attributes for the last defined track. The indention and the lower case letters are for readability only (format of the exporter). [key] is special, see below.
Instruction TRACK
Starts a new track definition.
Parameter varname
The name of a variable that will store the current slot number and track type as an integer. This name should then be used for the cup definitions. The value 0 means that no variable is created. Names beginning with bt or vs are reserved for this purpose.
Parameter type
The type of the track. Use either bt for a battle arena or vs versus track. Both are predefined constants.
Parameter property
The property slot of the track. Use either slot number (0-31 for versus tracks, 32-41 for battle arenas), or a symbolic name like T12, A25 or MMM. See chapter »Names of property and music slots« for details.
Parameter music
The music id of the track. Use either a music index between 175 and 200 or the name of a property slot. See chapter »Names of property and music slots« for details.
Parameter flags
Flags for the track. It is either a number build from LE$F_NEW, LE$F_HEAD or LE$F_GROUP, or a string that is scanned for letters »N«, »H« and »G« to build the number. See chapter »Track flags« for more details.
Instructions LAPS and SPEED
Define the number of laps or the speed factor of the track. It can be used for track listings or BMG output. If not set, it can be extracted from the attributes.
Instruction FILE
Define a filename for the track without trailing .szs. The filename is used when SZS files are copied, moved or links by instruction TRACKS=*.
Instruction IDENT
The identification. Recommended is the SHA1 or DB64 checksum of the unpacked SZS file, so that it can used for a SHA1 export. Some optimizations are planned for tracks with the same identification. An empty string or a single minus sign means that no identification is available.
Instruction NAME
The name of the track. Use it to build the in-game names.
Instruction XNAME
A second name of the track. Usually it is a longer variant of NAME with more info for external track listings.
Instruction [key]
Each track has its own string-set accessed by key (case sensitive). This command assigns the following string.

An example from MKW-Fun 2022-05

Here is an excerpt from MKW-Fun as an example. The full files are available here (battle arenas) and here (versus tracks).

Example: battle arenas

Example: versus tracks


Section [CUP-LIST]

This section defines how the arenas and tracks are distributed to the cups. Battle arenas+cups and versus tracks+cups are processed absolutely separately from each other.

Normally an author only determines the assignment of the standard tracks. All other tracks are then assigned to the cups in the order in which they were defined. Finally, if there is an odd number of cups, another cup is added, since there must always be an even number of cups. Cup slots that are not used or incorrectly used are automatically replaced by a filling track.

To achieve exactly this behavior, this minimal [CUP-LIST] section should be created: 

[CUP-LIST]
STANDARD-BATTLE-CUPS
STANDARD-VERSUS-CUPS
MKWFUN-RANDOM-CUP
ADD-UNUSED = yes

Analysis:

Section [CUP-LIST]
Start the cup definition section. The following instructions are only valid in this section.
Instruction STANDARD-BATTLE-CUPS
Append 8 tracks (slots 32 to 41) to the current battle cups in original order. If this is the first battle assignment, then the 2 original cups are defined. LE-CODE currently only supports 2 cups. Therefore, this instruction should not be used when defining custom arenas.
Instruction STANDARD-VERSUS-CUPS
Append 32 tracks (slots 0 to 31) to the current versus cups in original order. If this is the first versus assignment, then the 8 original cups are defined.
Instruction MKWFUN-RANDOM-CUP
Append the 4 random slots (62 to 65) that are supported by LE-CODE to the versus cups. The order is the same as MKW-Fun uses it. See chapter »Random tracks« for details.
Instruction ADD-UNUSED = no|yes
Define whether tracks that are not yet in a cup should be automatically assigned to a cup. yes is the recommended and default setting here. This setting belongs battle and versus cups. With instructions ADD-UNUSED-ARENAS and ADD-UNUSED-TRACKS (see below) this decision can be made separately.

There are some more instructions not used in the example:

Instruction APPEND name1 name2 ...
Append 1 or more tracks to the cups. The cup type (battle or versus) is automatically chhosen by the track type. Best is to use the variable names of instruction TRACK.
Instruction APPEND-BT name1 name2 ...
Same as APPEND, but only battle arenas and random tracks are added to the battle cups. Available as of SZS Tools v2.30a.
Instruction APPEND-VS name1 name2 ...
Same as APPEND, but only versus and random tracks are added to the versus cups. Available as of SZS Tools v2.30a.
Instruction NEW-CUP [bt] [vs]
Close the current cup and open a new one. Use either bt for a battle arena or vs versus track or both. This instruction is really never needed.
Instruction ADD-UNUSED-ARENAS = no|yes
Define whether battle arenas that are not yet in a cup should be automatically assigned to a battle cup.
Instruction ADD-UNUSED-TRACKS = no|yes
Define whether versus tracks that are not yet in a cup should be automatically assigned to a versus cup.

Add tracks by *.szs

wlect dis has various functions that do things automatically. A special function is the addition of SZS files to the track list. To do this, simply specify the SZS files on the command line after the LE-DEF file, e.g. 

wlect dis le-def.txt prefix.txt "./path/to/*.szs" ...

Analysis:

File le-def.txt
A LE-DEF file as initial setup.
File prefix.txt
An optional list with game prefixes that replaces is internal list. See chapter »Game prefixes« for details.
Files "./path/to/*.szs"
Scan track files and import attributes. Note the quotation marks. They prevent the evaluation of the wildcard * and thus keep the command line small (see chapter »Command line limits and solutions«). See below for a complete discussion about the import. See »How to enter ./path/to/…« for another discussion.

The name and the content of each SZS file is evaluated:

  • Only valid track files are accepted. Files of format *_d.szs are ignored.
  • The SHA1 is calculated and stored together with the filename and version number.
  • Property slot and Music-ID are calculated by content and filename attributes.
  • The NEW flag is set if filename attribute new is set or a plus-prefix is available.
  • Filename attributes head=NAME and grp=NAME are used to setup groups. Groups for battle arenas and versus tracks managed separately.
  • Filename attribute order=INT is used to setup a primary sort value. Values between −32768 and +32767 are allowed. The default is 1000.
  • The directory part of the SZS files is appended to the list of search directories for option --track-dir. The transfer mode can be set by option --szs-mode MODE. The default mode is LINK (like option --link-tracks).

The track information is first saved in a separate list. With the first instruction, this list is closed and evaluated as follows:

  • Groups are managed. Exact one track becomes group header, all other tracks are hidden group members.
  • Unhidden tracks are sorted on MKW fun method by attribute order, plus order, name without prefix, game order and track version. In a later version of the SZS tools, settings will determine the type of sorting.
  • Unhidden tracks are added to the main track list. If a group header is added, the all group members are added too.
  • If settings ADD-UNUSED-ARENAS and/or ADD-UNUSED-TRACKS are set, then the trackss are automatically inserted at the end of the cups.
  • The separate list is cleared.

Use command wlect dis "./path/to/*.szs" ledef=- to verify these automatic steps.

Creating or patching LE-CODE binaries

After creating the LE-DEF file le-def.txt and optionally the LPAR file lpar.txt you can use this command to create the LE-CODE binaries for all 4 regions in 1 step: 

wlect dis le-def.txt lpar.txt lecode=./outpath/[email protected]

With this command the built-in LE-CODE binaries were used and patched. Command wlect create le-info prints an information about the built-in binaries. See below for more details. If you want to use your own binaries (e.g more recent versions), just load them as first action: 

wlect dis +in-lecode ./inpath/lecode-*.bin le-def.txt lpar.txt lecode=./outpath/[email protected]

Analysis:

+in-lecode
Enable the import of LE-CODE binaries. If this processing option is disabled, only the settings will be imported.
./inpath/lecode-*.bin
Path to the LE-CODE binaries to be imported. See »How to enter ./path/to/…« for a discussion.
le-def.txt
The LE-DEF definition file.
lpar.txt
The LPAR definition file for general settings (optional).
lecode=./outpath/[email protected]
Output path for the 4 LE-CODE binaries. The last »@« character of the whole filename is replaced by »PAL«, »USA«, »JAP« and »KOR« in sequence.

Built-in and current version

Command wlect create le-info prints an information about the built-in LE-CODE binaries and command wlect dis ... le-info=- about the current LE-CODE binaries. At the beginning current and built-in are the same, but wlect dis ... can load external binaries.

Example: 

wlect dis le-info=- +in-lecode ./path/to/b28/lecode-*.bin le-info=-

Command analysis:

le-info=-
Print an info about the current (here built-in) versions to standard output.
+in-lecode
Enable loading of LE-CODE binaries.
./path/to/b30/lecode-*.bin
Load external LE-CODE binaries.See »How to enter ./path/to/…« for a discussion.
le-info=-
Print an info about the current (here loaded) versions to standard output.

Output


Copy, move and/or link track files

wlect DISTRIB supports an automated collecting and renaming of the track files. Therefor options --track-dir, --copy-tracks, --move-tracks, --move1-tracks and --link-tracks define the operation.

Option --track-dir DIRECTORY
With this option you define a destination directory, where all track files (type SZS) are copied, moved or linked to. Existing files with the same name are removed before without any notice. The usual destination directory is files/Race/Course/. The files are automatically renamed to 123.szs (and 123_d.szs), where »123« are the 3 hex digits representing the slot index.

Each filename of the definition file (option --def-lecode) is searched in the directories defined by the other 4 options in the order of the definition. Each option can be used multiple times and in any order to define any number of source directories. The option name itself defines only, how to copy the source file. After coping and renaming to 123.szs the main file, a _d file is searched too, and copied and renamed if found.

Option --copy-tracks DIRECTORY
Files found in this directory are copied to the destination directory. This is the recommended option, if you don't understand the concepts and the advantages of moving and linking files.
Option --move-tracks DIRECTORY
Files found in this directory are moved to the destination directory. If moving fails, the file is copied and the source removed.
Option --move1-tracks DIRECTORY
Files found in this directory are moved to the destination directory, but only if a file has not more than one hard link (no other usage). This guarantees an unique version of the file. If the file has more hard links or moving fails, the file is copied and the source removed.
Option --link-tracks DIRECTORY
Files found in this directory are hard linked to the destination directory. If linking fails, the file is copied.

It is possible, that the destination directory defined by option --track-dir is the same as one of the source directories. Here usually --move-tracks is the best choice to define it — the files are simply renamed.

Example for an usual command line (1 line only!): 

wlect dis --track-dir DESTDIR --copy-tracks SRCDIR1 --copy-tracks SRCDIR2 --move-tracks DESTDIR .... tracks=nolog

Analysis:

Option --track-dir DESTDIR
Define the destination directory for tracks. It is usually something like .../files/Race/Course/.
Option --copy-tracks SRCDIR1
First directory in which tracks are searched.
Option --copy-tracks SRCDIR2
Second directory in which tracks are searched.
Option --move-tracks DESTDIR
Also use the destination directory as a source to rename track files.
....
Any other arguments that .e.g. are described elsewhere in this article.
Instruction tracks=nolog
Copy, move and link files. Disable logging.

Create other files

It is possible to create many other files by adding more instructions to the command line. Call wlect dis for the built-in help to get a list of all instructions.

Example (1 command line): 

wlect dis -o .... xsha1=sha1.list distrib=distrib.txt +reset,custom names=names.txt \
                  xinfo=info.txt xrating=rating.txt +reset,lecode,names bmg=names.bmg

Analysis:

Global option -o
Allow overwriting existing files. It's a short cut for --overwrite.
....
Any other arguments that .e.g. are described elsewhere in this article.
Instruction xsha1=sha1.list
Export a SHA1 listing file sha1.list, that can be used at wiimmfi.de to define the allowed tracks for a region. Use XNAMES as source for the track names.
Instruction distrib=distrib.txt
Create a distribution info file, that can be imported by ct.wiimm.de. Write it to file distrib.bmg. It's like wszst distribution.
Processing options +reset,custom
Limit the following listing to custom tracks. Option reset prevents side effects of other previously set options.
Instruction names=names.txt
Write a human readable listing with cups and track names to file names.txt. Use NAMES as source for the track names.
Instruction xinfo=info.txt
Write a human readable listing with cups, slots, le-flags and track names to file info.txt. Use XNAMES as source for the track names.
Instruction xrating=rating.txt
Write a human readable listing with rating (parenthesis only) cups, slots and track names to file rating.txt. Use XNAMES as source for the track names. This is available as of Wiimms SZS Tools v2.29a.
Processing options +reset,lecode,names
Settings for the next BMG instructions. Define LE-CODE track names (message ids ≥7000) and use NAMES as source. Option reset prevents side effects of other previously set options.
Instruction bmg=names.bmg
Write all track names to BMG binary file names.bmg.

Notes

Names of property and music slots

The configuration file is read by a parser that support expression and more. For the property slot a value between 0 and 41 is expected. For the music-id a value 175 and 200 is expected. If a valid property value is read, than it is converted to the related music-id.

The property of Maple Treeway should be used as an example. The needed value is 11. Instead of the number, you can also use a predefined constant that is easier to remember. Two variants are defined for this:

  • The first is based on the slot number in the game. For battle arenas it is A11 to A25 and for versus tracks from T11 to T84.
  • Alternatively, abbreviations can be used for the track.

And so for our example Maple Treeway you can use the values T33 or MT. By the way, the names are case-insensitive. Study the table below for other tracks.

Property Slots and Music Ids
Slot Index Music ID Constant names Track name
— (117) 117 = 0x7f xLC Only a music id. The name can't be used for property slots.
Maybe it was planned for Luigi Circuit but not used by accident.
1.1 8 125 = 0x7d T11 LC Luigi Circuit
1.2 1 119 = 0x77 T12 MMM Moo Moo Meadows
1.3 2 121 = 0x79 T13 MG Mushroom Gorge
1.4 4 123 = 0x7b T14 TF Toad's Factory
2.1 0 125 = 0x7d T21 MC Mario Circuit
2.2 5 127 = 0x7f T22 CM Coconut Mall
2.3 6 129 = 0x81 T23 DKS DK Summit
2.4 7 131 = 0x83 T24 WGM Wario's Gold Mine
3.1 9 135 = 0x87 T31 DC Daisy Circuit
3.2 15 133 = 0x85 T32 KC Koopa Cape
3.3 11 143 = 0x8f T33 MT Maple Treeway
3.4 3 139 = 0x8b T34 GV Grumble Volcano
4.1 14 137 = 0x89 T41 DDR Dry Dry Ruins
4.2 10 141 = 0x8d T42 MH Moonview Highway
4.3 12 145 = 0x91 T43 BC Bowser's Castle
4.4 13 147 = 0x93 T44 RR Rainbow Road
5.1 16 165 = 0xa5 T51 gPB GCN Peach Beach
5.2 20 173 = 0xad T52 dYF DS Yoshi Falls
5.3 25 151 = 0x97 T53 sGV2 SNES Ghost Valley 2
5.4 26 159 = 0x9f T54 nMR N64 Mario Raceway
6.1 27 157 = 0x9d T61 nSL N64 Sherbet Land
6.2 31 149 = 0x95 T62 gSGB GBA Shy Guy Beach
6.3 23 175 = 0xaf T63 dDS DS Delfino Square
6.4 18 169 = 0xa9 T64 gWS GCN Waluigi Stadium
7.1 21 177 = 0xb1 T71 dDH DS Desert Hills
7.2 30 155 = 0x9b T72 gBC3 GBA Bowser Castle 3
7.3 29 161 = 0xa1 T73 nDKJP N64 DK's Jungle Parkway
7.4 17 167 = 0xa7 T74 gMC GCN Mario Circuit
8.1 24 153 = 0x99 T81 sMC3 SNES Mario Circuit 3
8.2 22 179 = 0xb3 T82 dPG DS Peach Gardens
8.3 19 171 = 0xab T83 gDKM GCN DK Mountain
8.4 28 163 = 0xa3 T84 nBC N64 Bowser's Castle
A1.1 33 183 = 0xb7 A11 aBP Block Plaza
A1.2 32 181 = 0xb5 A12 aDP Delfino Pier
A1.3 35 185 = 0xb9 A13 aFS Funky Stadium
A1.4 34 187 = 0xbb A14 aCCW Chain Chomp Wheel
A1.5 36 189 = 0xbd A15 aTD Thwomp Desert
A2.1 39 195 = 0xc3 A21 asBC4 SNES Battle Course 4
A2.2 40 197 = 0xc5 A22 agBC3 GBA Battle Course 3
A2.3 41 199 = 0xc7 A23 anSS N64 Skyscraper
A2.4 37 191 = 0xbf A24 agCL GCN Cookie Land
A2.5 38 193 = 0xc1 A25 adTH DS Twilight House

Track flags

Besides the standard properties property slot and music id, LE-CODE support a third property: flags. As of LE-CODE v5.b35 the flags are stored into 2 bytes (before into 1 byte). All three elements together occupy 16384 bytes.

flags is a bit field with following features, the bit numbers are or'ed:

New (Bit 0 = 0x01, symbol LE$F_NEW, character »N«)
This track is marked as NEW. It has only impact to the random selection at slot 0x041.
Random header (Bit 1 = 0x02, symbol LE$F_HEAD, character »H«)
Begin of a random group. The group ends, if either the Random group flag is not set or the next random group starts. If such track is selected (directly or by random), one of the following group members is selected by random. If the group header itself is group member too (both bits set), then it is included into the random selection; otherwise it is excluded.
Random group (Bit 2 = 0x04, symbol LE$F_GROUP, character »G«)
Track is member of the current random group. See Random header above.
Texture hack (Bit 4 = 0x10, symbol LE$F_TEXTURE, character »T«)
Track is a texture hack of a Nintendo track or arena. See section »Texture hacks for Nintendo tracks« for details.
Hidden (Bit 5 = 0x20, symbol LE$F_HIDDEN, character »I«)
The track is considered hidden and not included into cups. This flag is only used by Wiimms SZS Tools and other distribution generators, but ignored by LE-CODE. A track is also considered hidden if the group flag is set but the header flag is not.

Flags LE$F_TEXTURE and LE$F_HIDDEN are available as of LE-CODE build 34 & Wiimms SZS Tools v2.29a. Other bits are set by SZS Tools to support the random track selection.

Example: LE$F_HEAD|LE$F_NEW (value 3, string »NH«) means: Mark the track as new (LE$F_NEW) and start a random group (LE$F_HEAD). The track itself is not part of the random group (LE$F_GROUP is not set) and only used to display the track name.

Group of tracks

LE-CODE supports a certain type of groups, which are defined by the head=NAME and grp=NAME attributes when reading in *.szs files. The idea behind it: There is exactly 1 header track per group, which is displayed in the track selection. When choosing this track, one of the group members will be chosen at random. The header track can optionally itself be a group member. All tracks with the same NAME share the same group.

LE-CODE uses the flags LE$F_HEAD and LE$F_GROUP for this. The group members must be use the slots directly after the header. When reading in *.szs files, the wlect dis tool takes care of the correct arrangement. The following algorithm is used for this.

  • Groups are formed by their names. Battle arenas and versus tracks are considered separately.
  • If there is no or only one valid group member, then the group will be disbanded. Pure header tracks become invalid.
  • If there are several header tracks, only the first track will be accepted as such. Other header-only tracks become invalid.
  • If there is no header track for a group, then the first group member will automatically become the header.
  • Only header tracks are considered for track sorting. As soon as a header section is assigned to a slot, the remaining group members receive the other slots. Only track slots ≥68 are used for this.

Random texture hacks for Nintendo tracks

LE-CODE can already select tracks at random if they are grouped together. However, this method is not suitable for the original tracks and arenas when playing worldwide, since the slot assignment (from 0 to 41) must be retained. Therefore, LE-CODE offers a new feature from build 34 onwards.

Texture hacks can be stored for the original tracks and arenas from slot 68 onwards. The flag LE$F_TEXTURE must be set for this. Arenas should be hidden (not appear in a cup). Racing tracks can be optionally assigned to cups. If an original track is now selected and the new feature is enabled by LPAR settings BT-TEXTURES or VS-TEXTURES, then all tracks for the same property slot and with the flag LE$F_TEXTURE set are searched and one including or excluding the original track is selected at random. The original slot is always transmitted in the online protocol, which makes this feature compatible with the worldwide region.

With a little trick in random selection, the same track is selected for all players of the same distribution, provided this feature is activated for the player. But if blocking of texture hacks is enabled, the synchronization is broken for some situations. The distribution creator can allow players to turn the feature on or off by D-PAD Cheats.

Track Strings

Character strings can be specified for each individual track. A distinction is made between permanently existing character strings and dynamically definable character strings. The advantage of permanently existing character strings is the fixed storage location and thus quick access. There are also hardwired relationships.

In addition to the permanently defined strings, each track has a set of strings, where the strings are addressed via keys. Keys can consist of any character except »]« and are case-sensitive. Keys are always given in square brackets ([key]) to distinguish between the permanently defined strings. The strings are managed as an associative array. The user can freely dispose of this set of character strings. Some things are to be expanded on it later, e.g. a support for different languages.

The following table shows the names and explains the functionality of the character strings:

String Variables for each track
Name Type Descrption
ident perm The identification. Recommended is the SHA1 or DB64 checksum of the unpacked SZS file, so that it can used for a SHA1 export. Some optimizations are planned for tracks with the same identification. An empty string or a single minus sign means that no identification is available.
sha1 perm It's like identification, but only SHA1 and DB64 checksum are accepted on writing and only SHA1 checksums on reading. In fact, ident and sha1 are just different interfaces to the same memory space.
file perm The filename of the track. It is usually used to extract the individual components such as prefixes, version numbers or attributes.
name perm The simplified name of the track. Use it to build the in-game names.
xname perm A second name of the track. Usually it is a longer variant of NAME with more info for external track listings. If xname is not defined or empty, then name is used as fallback.
temp1
temp2		 perm 2 temporary variables with no specific purpose. Access is much faster than for dynamic strings. They are therefore intended for multi-level string manipulations.
[key] dyn Access to the dynamically definable character strings (associative array).

SHA1 and DB64

SHA1 (or SHA-1) is a checksum and hash function (Mediawiki). In Mario Kart Wii it is used for identifying and finding tracks. The checksum is calculated using the unpacked SZS file. This avoids different checksums due to different compressions. These checksums are supported by wiimmfi.de, ct.wiimm.de and CTGPR, among others. Wiimms SZS Tools also supports this type of checksum, including the commands wszst sha1 and wlect distribution (subject of this article).

A SHA1 checksum consists of 20 bytes of binary data. In order to make it readable, it is almost always output in hexadecimal with 40 characters. The command wszst sha1 supports various other output formats, including DB64 (option --db64). The 20 bytes of the checksum are appended with 4 bytes of the unpacked size (big endian), and the whole thing is then encoded with BASE64. This results in a length of 32 instead of 40 bytes for the checksum. DB64 was originally intended for the space-saving storage of checksums as text in a database.

If defining a track, an identification can be added by ident "TEXT". If TEXT is either a SHA1 or a DB64, then it is stored additionally as SHA1 checksum for the track. In the case of DB64, the SHA1 is extracted. The stored SHA1 is then used to generate some lists.

Game prefixes

When analyzing the track name, an attempt is made to determine the associated game or console. This prefix is then used later for sorting and color selection when displaying the track names.

This game prefix must appear after the optional plus prefix at the beginning of the name. Prefix combinations such as »Tour N64« are also recognized. An internal list that can be output by wlect create prefix is used for recognition. The original source of the list is ct.wiimm.de. If you download this file, you can replace the internal list with --load-prefix prefix.txt. With command wlect dis ..., this file can also be specified as input file without an option.

Plus prefixes

The Intermezzo uses special prefixes to divide the tracks into classes, which are also used for sorting. These plus-prefixes are always at the beginning of the track name and begin with a plus sign.

If a name begins with a plus sign, then all characters up to the first space are recognized as a plus prefix. The first part consists of all the plus signs followed by any other characters. The first character of the second part is used to determine the sort value. If the second part is empty, then the last plus sign is used instead.

The higher sorting value is now calculated from the number of plus signs in the first part, the more plus signs, the smaller the value. The first character from the second part determines the lower value. Option --plus chars defines a character list. If the first character is in chars then the position determines the value, otherwise the ASCII value to which 500 is added.

The default is + to give a plus sign the low value. Intermezzo uses 1+UVWT to get the following order: +++ +1 ++ +U +V +W +T +N.

Random track selection

LE-CODE reserved 5 slots for a special random selections. These slots can be added to versus cups by specifying their name or slot number. As of build 35 they can be used for racing tracks and for battle arenas. By using the keyword MKWFUN-RANDOM-SLOTS the 4 slots 62–65 can be added in one step. The internal algorithm changed to support arenas. However, you won't notice any change if the first 8 cups use the original tracks.

LE-CODE Random Slots
Slot
hex		 Slot
dec		 Var
Name		 Track Name Description
0x03d 61 rTex Random: Texture Hacks as of LE-CODE build 35 & Wiimms SZS Tools v2.29a
Select a track, that is marked by flag TEXTURE, by random.
0x03e 62 rAll Random: All Tracks Select a track, that is used by any cup, by random.
0x03f 63 rOrig Random: Original Tracks Select a track, that is used by the first 8 racing cups or by the 2 battle cups, by random.
0x040 64 rCust Random: Custom Tracks Select a track, that is used by any custom cup, by random.
0x041 65 rNew Random: New Tracks Select a track, that is marked by flag NEW, by random.

If no track of a random class is found, then an internal fallback function is used to select any an other existing track or arena. So all random selections are save, even if no track is defined for the related class.

Worldwide

Worldwide matches can be enabled for versus races (LPAR VS-WORLDWIDE) and/or for battle fights (LPAR BT-WORLDWIDE). If both are disabled, the Worldwide button of the online menu is hidden. If only one variant is activated, then the intermediate menu with the choice battle or versus is skipped. In this case, BMG message fa0 ("Worldwide") should be replaced by 106e ("Worldwide Battle") or 106d ("Worldwide VS Race"). Futrure versions of LE-CODE will do it automatically. If both variants are active, the intermediate menu is displayed as usual.

In addition to the settings, the following things must be ensured:

The following special traits apply to worldwide battles and versus races:

The last 5 points are not absolutely necessary, since the original arenas and tracks do not use these properties. However, deactivation reduces the risk of cheats.

Filling tracks

After defining cups, some cup slots may still be unoccupied and/or invalid. In this case, the following algorithm is applied. This happens separately for battle and versus cups:

  • Terminate this algorithm if all cups are occupied with valid tracks.
  • If no fill tracks is defined yet, clone an already defined track of the same type and name it »−« (single minus sign). Mark this new track as a filling track .
  • Fill in the track index in all invalid and unoccupied cup slots.

The word »fill« appears in cup listings. Filling tracks are not displayed in track lists. When importing the SZS files with option --track-dir, after all other tracks have been transferred, a link (or a copy if linking failed) of the track that was cloned is created.

This means that all cup slots are occupied and the track can be selected and played.

How to enter ./path/to/...

Unix-based operating systems (e.g. Linux, BSD, MacOS) and other operating systems (e.g. CP/M, OS/2) use the format /path/to/file for absolute paths and path/to/file or ./path/to/file or ../path/to/file for relative paths to files. Windows, on the other hand, uses e.g. A:\path\to\file or ..\\path\\to\\file for such paths.

And here the problems begin. The backslash \ introduced an escape sequence, e.g. to describe \n for a NewLine character. Under Windows, this leads to problems due to the lack of clarity. Depending on the tool you have to write .\path\to\file or .\\path\\to\\file. However, the DOS and Windows APIs always support the forward slash (/) as a directory separator, only the batch interpreters for *.bat and *.cmd files don't use this ability.

Unix-based operating systems don't support a leading C:\\ for a different file system. In these operating systems, other file systems are attached somewhere in the path. If a path starts with a /, then it is an absolute path based on the root directory. Otherwise, they are relative paths to the current directory.

Wiimm develops his tools under Linux. To make them available on Windows he uses Cygwin, a Linux like programming and runtime environment that runs natively on Microsoft Windows (Wikipedia). The Cygwin variants of the tools support both Unix and Windows notation for paths and files. The latter is translated into Linux notation, with specifications such as C:\\ being translated into /cygdrive/c/. If using command 

wszst expand  C:\\path\\to\\file  C:/path/to/file  \\cygdrive\\c\\path\\to\\file  /cygdrive/c/path/to/file

… you will see, that all 4 paths results in the same /cygdrive/c/path/to/file.

Usually you will use relative paths. For example, if your tracks are in a directory tracks below the working (current) directory, than you will use "./tracks/*.szs" or "tracks/*.szs". The leading ./ is just cosmetic to indicate the relative path. The quotation marks ensure that the SZS tools search for the files (see »Command line limits and solutions«).

Supported file types

The list below shows all file types accepted by wlect dis. The Name column shows the name returned by command * filetype (available in all tools). Type is either text or binary. Magic specifies the string by which the file type is recognized. If via is set to CT-CODE, the file is first read in via the CT-CODE scanner and then imported to the internal data model.

Supported file formats
Name Type Magic via Description
LE-DEF text #LE-DEF1 – A LE-CODE definition file. It defines tracks and cups.
LE-DIS text #LE-DIST – A dump file, that contains all data of the internal data model.
LE-REF text #LE-REF1 – A dump file, that contains a track list of the internal data model.
LE-STR text #LE-STR1 – A dump file, that contains all track strings of the internal data model.
SHA1REF text #SHA1REF – A dump file, that contains all SHA1 checksums. Can be imported by wiimmfi.de.
LE-BIN binary LECT – A LE-CODE binary file, that is used to patch the executor.
MTCAT text #MTCAT03 – A file to define MKW track categories. ct.wiimm.de is the authoritative source for it.
PREFIX text #PREFIX1 – A prefix file to identify the game prefixes. ct.wiimm.de is the authoritative source for it.
LPAR text #LE-LPAR – A LE-CODE parameter file.
BMG binary MESGbmg1 – A BMG message file. Import and export of strings is possible.
BMG-TXT text #BMG – Like BMG, but the text variant.
DISTRIB text #DISTRIB – A distribution information file. Can be imported by ct.wiimm.de.
U8 (SZS) binary 55 aa 38 2d – A track file. The U8 content is usually compressed and stored as SZS file.
CT-DEF text #CT-CODE CT-CODE A CT-CODE definition file.
BRRES binary bres CT-CODE A CT-CODE binary file of type BRRES with embedded CT definition (C1DATA).
TEX+CT binary TEX0 CT-CODE A CT-CODE binary file of type TEX0 with embedded CT definition (C1DATA).
C1DATA binary ba d1 da 7a CT-CODE Main CT-CODE data.
Retrieved from "https://wiki.tockdom.com/w/index.php?title=LE-CODE/Distribution_Tutorial/LE-DEF&oldid=373864"