View unanswered posts | View active topics It is currently Fri Mar 29, 2024 12:26 am



Reply to topic  [ 22 posts ]  Go to page 1, 2  Next
 TWX Technical Discussion (and decompiling info) 
Author Message
Lieutenant
User avatar

Joined: Tue Dec 17, 2002 3:00 am
Posts: 516
Location: Virginia
Unread post TWX Technical Discussion (and decompiling info)
Hi everyone,

I am preparing to (soon-ish) share my cts decompiler with the world -- many people have been asking, and I did promise to release it, though I had to fix some bugs first.

Along the way, I have learned many things about the internals of how twx works -- a lot of it related to how it processes and handles data and how the scripting language works, but also some about the proxy itself. Micro has been doing great work in updating the proxy, and I am sure has things to contribute here as well.

I'm planning to post a series of articles that go into what I've discovered, so that it is here for prosperity -- if I lose the time to actively contribute (or anyone else does), I am hoping this will help the community. I would love for others who have in-depth knowledge to contribute as well, because I fully admit that there is still a lot I don't personally know.

_________________
TOURNAMENT WINNER: ICE 2017 - ICE 2019 - SUMMER SPLASH 2019 - XMAS TOURNEY 2019


Tue Oct 01, 2019 10:31 pm
Profile ICQ
Lieutenant
User avatar

Joined: Tue Dec 17, 2002 3:00 am
Posts: 516
Location: Virginia
Unread post Re: TWX Technical Discussion (and decompiling info)
Okay, my first prepared article -- since I am still polishing up my other notes, I will jump into this now, since it's got some interesting tidbits.

This is about some of the unusual or surprising things I learned during the process -- specifically about how certain things you do in scripts are handled in the actual proxy itself. Feedback or additional thoughts welcome.

========================================================================================

1. MERGING TEXT

A lot of people believe that the following two lines of code do the same thing in TWX (and would be forgiven for thinking that, since the reasons why they don't are not obvious):

a) echo "Output test " & $testnum & " of " & $testcount & "*"
b) echo "Output test " $testnum " of " $testcount "*"

They are actually compiled very differently.

Line (a) is translated into the following system commands (this output is from my decompiler debug function):

Code:
<S0> <L1> ---START COMMAND---
<S0> <L1> COMMAND: MERGETEXT
<S0> <L1> PARAM_VAR: $TESTCOUNT
<S0> <L1> PARAM_CONST: "*"
<S0> <L1> PARAM_VAR: $$7
<S0> <L1> MERGED $$7 = $TESTCOUNT & "*"
<S0> <L1> ---END COMMAND---
<S0> <L1> ---START COMMAND---
<S0> <L1> COMMAND: MERGETEXT
<S0> <L1> PARAM_CONST: " of "
<S0> <L1> PARAM_VAR: $$7
<S0> <L1> PARAM_VAR: $$5
<S0> <L1> FOUND VAR: $$7 = $TESTCOUNT & "*"
<S0> <L1> MERGED $$5 = " of " & $TESTCOUNT & "*"
<S0> <L1> ---END COMMAND---
<S0> <L1> ---START COMMAND---
<S0> <L1> COMMAND: MERGETEXT
<S0> <L1> PARAM_VAR: $TESTNUM
<S0> <L1> PARAM_VAR: $$5
<S0> <L1> PARAM_VAR: $$3
<S0> <L1> FOUND VAR: $$5 = " of " & $TESTCOUNT & "*"
<S0> <L1> MERGED $$3 = $TESTNUM & " of " & $TESTCOUNT & "*"
<S0> <L1> ---END COMMAND---
<S0> <L1> ---START COMMAND---
<S0> <L1> COMMAND: MERGETEXT
<S0> <L1> PARAM_CONST: "Output test "
<S0> <L1> PARAM_VAR: $$3
<S0> <L1> PARAM_VAR: $$1
<S0> <L1> FOUND VAR: $$3 = $TESTNUM & " of " & $TESTCOUNT & "*"
<S0> <L1> MERGED $$1 = "Output test " & $TESTNUM & " of " & $TESTCOUNT & "*"
<S0> <L1> ---END COMMAND---
<S0> <L1> ---START COMMAND---
<S0> <L1> COMMAND: ECHO
<S0> <L1> PARAM_VAR: $$1
<S0> <L1> FOUND VAR: $$1 = "Output test " & $TESTNUM & " of " & $TESTCOUNT & "*"
<S0> <L1> ---END COMMAND---


Line (b) looks like this:

Code:
<S0> <L3> ---START COMMAND---
<S0> <L3> COMMAND: ECHO
<S0> <L3> PARAM_CONST: "Output test "
<S0> <L3> PARAM_VAR: $TESTNUM
<S0> <L3> PARAM_CONST: " of "
<S0> <L3> PARAM_VAR: $TESTCOUNT
<S0> <L3> PARAM_CONST: "*"
<S0> <L3> ---END COMMAND---


So to summarize, using (a) requires the twx program to execute the following internal commands:

MERGETEXT $TESTCOUNT "*" $$7
MERGETEXT " of " $$7 $$5
MERGETEXT $TEXTNUM $$5 $$3
MERGETEXT "Output test " $$3 $$1
ECHO $$1

While (b) results in the following command:

ECHO "Output test " $testnum " of " $testcount "*"

2. The use of "WAITFOR" versus "WAITON"

One is faster! No, it's the other! Never ever use "waitfor", it's bad!

I've heard all of this from various people over the years, so when I started digging into the released TWX delphi code, this was one of the first things I researched. What I found surprised me, and was confirmed once I had a working decompiler.

The following code:

waiton "waiton_test"
waitfor "waitfor_test"

Decompiles to this (after being compiled):

settexttrigger WAITON1 :WAITON1 "waiton_test"
pause
:WAITON1
waitfor "waitfor_test"

You're probably thinking "huh, what?" Yes -- "waiton" is simply a hard coded macro for a trigger. And it's actually worse than that -- it always calls the triggers WAITON1, WAITON2, etc, within a given namespace, so there can potentially be problems with things like merging scripts together using includes.

Let's look at how the code compiles.

(a) waiton "waiton_test"

Code:
<S0> <L1> ---START COMMAND---
<S0> <L1> COMMAND: SETTEXTTRIGGER
<S0> <L1> PARAM_CONST: "WAITON1"
<S0> <L1> PARAM_CONST: :WAITON1
<S0> <L1> PARAM_CONST: "waiton_test"
<S0> <L1> ---END COMMAND---
<S0> <L1> ---START COMMAND---
<S0> <L1> COMMAND: PAUSE
<S0> <L1> ---BEGIN END LABELS---
<S0> <L1> LABEL FOUND: WAITON1
<S0> <L1> LABEL PARSED: WAITON1
<S0> <L1> ---FINISH END LABELS---
<S0> <L1> ---END COMMAND---


(b) waitfor "waitfor_test"

Code:
<S0> <L2> ---START COMMAND---
<S0> <L2> COMMAND: WAITFOR
<S0> <L2> PARAM_CONST: "waitfor_test"
<S0> <L2> ---FINISH END LABELS---
<S0> <L2> ---END COMMAND---


Okay, so the waitfor definitely compiles smaller and takes less cycles to execute, but what functions does it call within the code, is there a difference there? So, let's take a look.

(a) waiton "waiton_test"

First, let's see how the triggers are defined/instantiated:

Code:
public static ScriptRef.TCmdAction CmdSetTextTrigger(Object Script, ScriptRef.TCmdParam[] __Params)
{
    ScriptRef.TCmdAction result;
    string Value;
    // CMD: setTextTrigger <name> <label> [<value>]
    if ((__Params.Length < 3))
    {
   Value = "";
    }
    else
    {
   Value = __Params[2].Value;
    }
    ((TScript)(Script)).SetTextTrigger(__Params[0].Value, __Params[1].Value, Value);
    result = ScriptRef.TCmdAction.caNone;
    return result;
}


Okay, SetTextTrigger is the function that is called for each, which looks like this:

Code:
public void SetTextTrigger(string Name, string LabelName, string Value)
{
    FTriggers[ttText].Add(CreateTrigger(Name, LabelName, Value));
}


And ultimately those triggers are checked in TextOutEvent (part of the "process out" stage of text handling):

Code:
public bool TextOutEvent(string Text, ref bool Handled)
{
    bool result;
    FOutText = Text;
    // check through textOut triggers for matches with text
    result = CheckTriggers(FTriggers[ttTextOut], Text, true, false, ref Handled);
    return result;
}

public static ScriptRef.TCmdAction CmdProcessOut(Object Script, ScriptRef.TCmdParam[] __Params)
{
    ScriptRef.TCmdAction result;
    // CMD: processOut <text>
    if (!(GlobalUnit.TWXInterpreter.TextOutEvent(__Params[0].Value, ((TScript)(Script)))))
    {
   GlobalUnit.TWXClient.Send(__Params[0].Value);
    }
    result = ScriptRef.TCmdAction.caNone;
    return result;
}


Okay, so triggers (all of them) get handled by PROCESSOUT. Which is interesting.

(b) waitfor "waitfor_test"

Here we define the waitfor in this function:

Code:
public static ScriptRef.TCmdAction CmdWaitFor(Object Script, ScriptRef.TCmdParam[] __Params)
{
    ScriptRef.TCmdAction result;
    // CMD: waitFor <value>
    ((TScript)(Script)).WaitText = __Params[0].Value;
    ((TScript)(Script)).WaitForActive = true;
    result = ScriptRef.TCmdAction.caPause;
    return result;
}


This gets scanned in the TextEvent function, which also processes triggers, but notably after the FWaitForActive value:

Code:
public bool TextEvent(string Text, bool ForceTrigger)
{
    bool result;
    bool Handled;
    // check waitfor
    if (FWaitForActive)
    {
   if ((Text.IndexOf(FWaitText) > 0))
   {
       FTriggersActive = false;
       FWaitForActive = false;
       result = Execute();
       return result;
   }
    }
    // check through textTriggers for matches with Text
    result = CheckTriggers(FTriggers[ttText], Text, false, ForceTrigger, ref Handled);
    return result;
}

public static ScriptRef.TCmdAction CmdProcessIn(Object Script, ScriptRef.TCmdParam[] __Params)
{
    ScriptRef.TCmdAction result;
    // CMD: processIn processType <text>
    if ((__Params[0].Value == '1'))
    {
   // process globally for all scripts
   ((TScript)(Script)).Controller.TextEvent(__Params[1].Value, true);
   ((TScript)(Script)).Controller.TextLineEvent(__Params[1].Value, true);
    }
    else
    {
   // process locally only
   ((TScript)(Script)).TextEvent(__Params[1].Value, true);
   ((TScript)(Script)).TextLineEvent(__Params[1].Value, true);
    }
    result = ScriptRef.TCmdAction.caNone;
    return result;
}


So, my thoughts on this topic:

a) It does seem that waitfor() is a simpler function that requires generally less processing cycles and does not have the risk of being prioritized against or bumped by other triggers.

b) Notably, waitfor() is ONLY processed on the PROCESSIN side, while waiton() [as a trigger] is processed at both PROCESSIN and PROCESSOUT. There are definitely implications here, though I am not sure what all of them are -- text should not be changing in a meaningful way between the two but could something get dropped by waitfor() that might be seen by waiton()? Or is waitfor() going to always be faster because it uses less cycles to process and it hits immediately when text comes in (before waiton)?

_________________
TOURNAMENT WINNER: ICE 2017 - ICE 2019 - SUMMER SPLASH 2019 - XMAS TOURNEY 2019


Tue Oct 01, 2019 10:40 pm
Profile ICQ
Lieutenant
User avatar

Joined: Tue Dec 17, 2002 3:00 am
Posts: 516
Location: Virginia
Unread post Re: TWX Technical Discussion (and decompiling info)
Okay, second article. Probably should have been the first, but I was still writing it, so...

Shadow's Technical Notes on Compiled TS (CTS) scripts
October 1, 2019
========================================================================================

1. OVERVIEW (What is a CTS anyway?)

Short answer: a .cts file is compiled bytecode that can be loaded and executed quickly by TWX Proxy. It is not an encrypted .ts file (in fact, there is no encryption done in the compilation process.)

Longer answer:

TWX Proxy reads and executes scripts in either .ts (twx script) or .cts (compiled twx script) format. Any .ts script can be compiled to a .cts using TWXC.EXE, shipped with the proxy distribution.

The primary benefit of a .cts file is that it loads faster because it is compiled bytecode that can be immediately processed into a Script object and executed. When loading a .ts file with twx, the file is first compiled (internally) and then processed into a Script object. Using a pre-compiled .cts file saves the compilation time and results in a much faster script starting time. It does NOT, however, have any effect on the speed of the script once it starts, since both are compiled before being executed.

2. WHAT'S NOT IN A CTS FILE?

While much of the information in the original TS file is maintained in the CTS output, some data is lost during the compilation to bytecode, and is impossible to restore through decompiling. Specifically, this includes:

a) Comments. This one is particularly frustrating if you are trying to understand how something works (assuming it was commented originally), but it is part of the process - the compiler simply throws out any line that starts with "#" and does not compile it in to the CTS.

b) Formatting. When converting all commands and parameters to bytecode, the compiler ignores any extra spaces, tabs, and things like extra parentheses around statements. These cannot be restored by the decompiler, though I have added an indent option that will indent using a standard method with tabs (if enabled), to make the code more readable.

c) Filesystem structure (for includes). This one is probably not an issue if you are decompiling single .ts files that do not call includes. However, for includes, the CTS only contains the filename of the included script and not its original location; so the decompiler will drop the includes in the same directory as the original script.

As a side effect of the lack of file structure, if you happen to include scripts with the same name from different directories, you have to either append the includes into a single file or create multiple files. For example, if you have the following includes in your script:

include "source\bot_includes\player\quikstats\player"
include "source\bot_includes\player\currentprompt\player"
include "source\bot_includes\player\startcnsettings\player"
include "source\bot_includes\player\getinfo\player"
include "source\bot_includes\player\moveintosector\player"
include "source\bot_includes\ship\getshipcapstats\ship"
include "source\bot_includes\ship\getshipstats\ship"
include "source\bot_includes\ship\savetheship\ship"
include "source\bot_includes\ship\loadshipinfo\ship"

This gets stored in the CTS file header as the following list of includes (note that the base script retains its full name with extension):

MOMBOT.TS
PLAYER
PLAYER
PLAYER
PLAYER
PLAYER
SHIP
SHIP
SHIP
SHIP

This produces the side effect that you have to either append files with the same namespace into a single file per namespace, or leave everything in the original ts file. If you don't intend to create a new directory structure for the includes (or move them back to one), it is necessary to name them the same for the code to work because the namespace that is used for includes is based on the filename -- so a variable called "test" in "player.ts" is referenced as "$player~test". I chose, in my decompiler, to append the files to a single include in the same directory as the base script, to keep a clean namespace.

3. FILE STRUCTURE

Each .cts file has the following overall structure:

Header (twxscript version[1-5], description size, file size)
Description
Parameters (all of the parameters used by the bytecode -- strings, values, etc)
Labels (for gosub/goto, if/else/elseif/while)
Includes (a list of all included files/namespaces)
Bytecode (a series of commands, parameters, and labels)

More details on each are as follows:

(a) Header

The header is defined in "ScriptCmp.pas" in the Delphi release of TWX. My port to C# is as follows:

Code:
// header at top of compiled script file (from ScriptCmp.pas)
[StructLayout(LayoutKind.Explicit)]
public struct TScriptFileHeader
{
   [FieldOffset(0)]
   [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 12)]
   public string ProgramName;

   [FieldOffset(12)]
   public ushort Version;

   [FieldOffset(16)]
   public int DescSize;

   [FieldOffset(20)]
   public int CodeSize;
}


Version is one of the following, mostly used for backward compatibility (to know what versions can be read by each version of twx):

// TWX Proxy 2.02 is version 1
// TWX Proxy 2.03Beta is version 2?
// TWX Proxy 2.03Final is version 3
// TWX Proxy 2.04 is version 4
// TWX Proxy 2.05 is version 5

The size offsets are used when reading the cts file, to know how many bytes to read for the description and bytecode.

(b) Description

Description allows for the contents of an additional description file to be appended to the script. This is mostly unused today.

(c) Parameters

Immediately following the description are the parameters. Parameters are used by nearly all of the commmands in the bytecode. Paramaters are stored as one of the following types:

Code:
public const int PARAM_VAR = 1;
// User variable prefix
public const int PARAM_CONST = 2;
// Compiler string constant prefix
public const int PARAM_SYSCONST = 3;
// Read only system value
public const int PARAM_PROGVAR = 4;
// Program variable
public const int PARAM_CHAR = 5;
// Single character value


Each parameter has a type, a length and a value. They are stored in order, at the beginning of the code section of the file, and must be read and stored and then dereferenced while processing the bytecode.

The first type, TCmdParam, are command parameters. The second type, TVarParam, are variable names.

Here is an example of some CmdParams:

Code:
CmdParam = You have a corporate memo from
CmdParam = RELOG
CmdParam = :CONNECTIVITY~KEEPALIVE
CmdParam = CONNECTION LOST
CmdParam = ONLINE_WATCH
CmdParam = :CONNECTIVITY~ONLINE_WATCH
CmdParam = Your session will be terminated in
CmdParam = KEEPALIVE
CmdParam = :CONNECTIVITY~KEEPALIVE
CmdParam = 30000


And some VarParams:

Code:
VarParamName = $BOT~UNLIMITEDGAME
VarParamName = $~UNLIMITEDGAME
VarParamName = $SHIP~CAP_FILE
VarParamName = $PLANET~PLANET_FILE
VarParamName = $GAME~MBBS
VarParamName = $BOT~_CK_PTRADESETTING
VarParamName = $BOT~RYLOS
VarParamName = $BOT~ALPHA_CENTAURI
VarParamName = $BOT~STARDOCK


The code to read the parameters, in C#, is as follows:

Code:
ParamType = br.ReadByte();
while (ParamType > 0)
{
   if ((ParamType == 1))  // TCmdParam (1)
   {
      Len = br.ReadInt32();
      Val = br.ReadBytes(Len);
      TCmdParam Param = new TCmdParam();
      ValStr = System.Text.Encoding.Default.GetString(Val);
      Param.Value = ApplyEncryption(ValStr, 113);
      FParamList.Add(Param);
   }
   else   // TVarParam (2)
   {
      Len = br.ReadInt32();
      byte[] PVal = br.ReadBytes(Len);
      TVarParam Param = new TVarParam();
      String PValStr = System.Text.Encoding.Default.GetString(PVal);
      Param.Value = ApplyEncryption(PValStr, 113);
      Len = br.ReadInt32();
      byte[] PName = br.ReadBytes(Len);
      String PNameStr = System.Text.Encoding.Default.GetString(PName);
      Param.Name = ApplyEncryption(PNameStr, 113);
      FParamList.Add(Param);
   }
   ParamType = br.ReadByte();
}


You will note the "ApplyEncryption" functions used here. There is a very very basic "encryption" done on all command parameters -- which includes all text values -- in the cts file. It's really more aptly described as "obfuscation" than encryption, but getting this to decrypt properly took some time.

One of the other challenges in reading the parameters it that there is no length provided for the parameters section; it terminates with a ParamType of 0, and this is the only way to know when you've reached the end (so if you are not checking the length of each value you are reading, you will over-read this section and miss the next.)

(d) Includes

Some scripts include other files, as mentioned above. This section is a simple list of all included files. Like the parameters, it ends with a null.

Reading it is very easy:

Code:
Len = br.ReadInt32();

while (Len > 0)
{
   Val = br.ReadBytes(Len);
   String inc = Encoding.UTF8.GetString(Val, 0, Val.Length);
   IncludeList.Add(inc);
   Len = br.ReadInt32();
}


These include file names are very important later if you want to split the include files back out, because that requires processing any variables or labels that include $INCLUDE~ or INCLUDE~: strings.

(e) Labels

The labels section is next in the cts file. Labels designate "code branches" and are used by the GOTO/GOSUB functions as well as by triggers.

In addition to the labels created by the script writer, the compilation process creates additional labels that it uses to track branches. I will address the whole topic of branches in a different article.

Labels have a value and a location, and look like this:

Code:
Label = CHECKSTARTINGPROMPT (Location 1556)
Label = KILLTHETRIGGERS (Location 1856)
Label = :17 (Location 1607)
Label = :18 (Location 1607)
Label = BOT~:19 (Location 1850)
Label = BOT~:20 (Location 1850)
Label = BOT~BIGDELAY_KILLTHETRIGGERS (Location 1889)
Label = BOT~UNFREEZEBOT (Location 1922)
Label = BOT~WAIT_FOR_COMMAND (Location 1966)


The labels with a ~ in them refer to include files, which I will also deal with in another article.

The reason for the location - and this might be the least fun part of writing a decompiler for cts files - is that the labels are not in order like the parameters, but instead have a location (in bytes from the start of the bytecode section) and must be inserted in the output file at the given offset while processing the bytecode.

(f) Bytecode

Finally, the bytecode section. This contains the most information, and is surprisingly simple (for fast execution), but reading and processing this is the most complex part of decompiling a script; the code for processing the bytecode alone, in C#, is over 1200 lines.

The bytecode is read, byte by byte, until the end is reached (as defined in the header). This starts out as follows:

Code:
// fetch script ID
byte ExecScriptID = CodeRef.ReadByte();
// fetch line number
CodeLine = (int)CodeRef.ReadUInt16();
// fetch command from code
CmdLine.ID = CodeRef.ReadUInt16();
ScriptCmd Cmd = TCmd.Cmds(ref CmdLine.ID);
CmdLine.Name = Cmd.Name;


After the bytecode header, a list of parameter references follows, terminated by a null. Each reference points to an integer that refers back to the parameter number (from the previously red CmdParams). In some cases, references can point to more than one parameter. I will write more detail on this in another article.

_________________
TOURNAMENT WINNER: ICE 2017 - ICE 2019 - SUMMER SPLASH 2019 - XMAS TOURNEY 2019


Wed Oct 02, 2019 12:15 am
Profile ICQ
Ambassador
User avatar

Joined: Wed Apr 20, 2011 1:19 pm
Posts: 2559
Location: Oklahoma City, OK 73170 US
Unread post Re: TWX Technical Discussion (and decompiling info)
I have been updating the TWX WIKI...

What? theyre is a TWX WIKI???

Yes there is a TWX WIKI... Well there is now... I have been working on it for a month or so...

I added your comments to:
https://github.com/MicroBlaster/TWXProxy/wiki/echo

and:
https://github.com/MicroBlaster/TWXProxy/wiki/waitFor

I have to agree with the adage "Never ever use 'waitfor', it's bad!" It may be faster, but I don't think you will get the results you are looking for. The "wait-on" macro isn't actually a command, it quite literally creates a text trigger. You will get a valid "CURRENT_LINE" which doesn't seem to be true for waitFor.

_________________
Regards,
Micro

Website: http://www.microblaster.net
TWGS2.20b/TW3.34: telnet://twgs.microblaster.net:2002

ICQ is Dead Jim! Join us on Discord:
https://discord.gg/zvEbArscMN


Wed Oct 02, 2019 12:26 am
Profile ICQ YIM WWW
Ambassador
User avatar

Joined: Wed Apr 20, 2011 1:19 pm
Posts: 2559
Location: Oklahoma City, OK 73170 US
Unread post Re: TWX Technical Discussion (and decompiling info)
Shadow wrote:
// TWX Proxy 2.02 is version 1
// TWX Proxy 2.03Beta is version 2?
// TWX Proxy 2.03Final is version 3
// TWX Proxy 2.04 is version 4
// TWX Proxy 2.05 is version 5

and...

// TWX Proxy 2.06 is version 6

_________________
Regards,
Micro

Website: http://www.microblaster.net
TWGS2.20b/TW3.34: telnet://twgs.microblaster.net:2002

ICQ is Dead Jim! Join us on Discord:
https://discord.gg/zvEbArscMN


Wed Oct 02, 2019 12:30 am
Profile ICQ YIM WWW
Lieutenant
User avatar

Joined: Tue Dec 17, 2002 3:00 am
Posts: 516
Location: Virginia
Unread post Re: TWX Technical Discussion (and decompiling info)
Micro wrote:
I have been updating the TWX WIKI...

What? theyre is a TWX WIKI???

Yes there is a TWX WIKI... Well there is now... I have been working on it for a month or so...

I added your comments to:
https://github.com/MicroBlaster/TWXProxy/wiki/echo

and:
https://github.com/MicroBlaster/TWXProxy/wiki/waitFor


I did not know that, thank you, I will look. I am planning to resume serious work on the full twx port to c# soon, so I am sure I will have a lot to add.

Quote:
I have to agree with the adage "Never ever use 'waitfor', it's bad!" It may be faster, but I don't think you will get the results you are looking for. The "wait-on" macro isn't actually a command, it quite literally creates a text trigger. You will get a valid "CURRENT_LINE" which doesn't seem to be true for waitFor.


Okay, I hear you and I realize it creates a trigger, but I would like more data behind "never use it, it's bad". Why? :) I know you're not really a Delphi programmer either (is anyone?) but looking through the code earlier, it was not completely obvious to me why it would be bad, other than possibly blocking triggers (since it pauses processing them until the waitfor is reached).

Incidentally, I forgot to mention in that first article that there are some cases where you have to use "&" instead of just inserting variables in an output string. You have to do this any time the params are treated as a static value, such as if you are using them with triggers. Eg:

settextrtrigger test :test "My name is "&$bot_name

works, but

settextrtrigger test :test "My name is " $bot_name

does not.

If you don't have to use the mergetext (&) command, though, it's faster not to.

_________________
TOURNAMENT WINNER: ICE 2017 - ICE 2019 - SUMMER SPLASH 2019 - XMAS TOURNEY 2019


Last edited by Shadow on Wed Oct 02, 2019 12:33 am, edited 1 time in total.



Wed Oct 02, 2019 12:30 am
Profile ICQ
Ambassador
User avatar

Joined: Wed Apr 20, 2011 1:19 pm
Posts: 2559
Location: Oklahoma City, OK 73170 US
Unread post Re: TWX Technical Discussion (and decompiling info)
I only know that when I have used wait-for it has not done what I expected.

_________________
Regards,
Micro

Website: http://www.microblaster.net
TWGS2.20b/TW3.34: telnet://twgs.microblaster.net:2002

ICQ is Dead Jim! Join us on Discord:
https://discord.gg/zvEbArscMN


Wed Oct 02, 2019 12:32 am
Profile ICQ YIM WWW
Lieutenant
User avatar

Joined: Tue Dec 17, 2002 3:00 am
Posts: 516
Location: Virginia
Unread post Re: TWX Technical Discussion (and decompiling info)
Micro wrote:
I only know that when I have used wait-for it has not done what I expected.


Okay. At some point I'll try to look into it.

If we know for certain that waitfor() is bad, then you should modify twx to just auto convert waitfor() to waiton(). But if there might be some instances where it's preferred to use waitfor, then we should leave it in.

_________________
TOURNAMENT WINNER: ICE 2017 - ICE 2019 - SUMMER SPLASH 2019 - XMAS TOURNEY 2019


Wed Oct 02, 2019 12:43 am
Profile ICQ
Commander
User avatar

Joined: Fri Jun 09, 2006 2:00 am
Posts: 1396
Location: Canada
Unread post Re: TWX Technical Discussion (and decompiling info)
Shadow wrote:
If we know for certain that waitfor() is bad, then you should modify twx to just auto convert waitfor() to waiton(). But if there might be some instances where it's preferred to use waitfor, then we should leave it in.


Firstly: Wow. You’ve obviously been very busy.

If I could chime in something for consideration, re WaitForing in TWX. The Script.HTML
makes a Note about WaitFor (which I've not noticed before):

Notes: If a trigger activates while this command is active, the command is immediately forgotten.

There is no reason why this command should be used instead of a trigger, except for the fact that it is easier to manage.


....which might explain some bugs I've experienced in the past.

Great discussion. Thanks for sharing.

_________________
----------------------------
-= QUANTUM Computing 101: 15 = 3 x 5 ... 48% of the time.
-= There are 10 types of people in the world: Those that understand Binary and those who do not
-= If Oil is made from Dinosaurs, and Plastic is made from Oil... are plastic Dinosaurs made from real Dinosaurs?
-= I like to keep my friends and my enemies rich, and wait to see which is which - Tony Stark (R.I.P.)


Wed Oct 02, 2019 7:00 am
Profile ICQ YIM
Lieutenant
User avatar

Joined: Tue Dec 17, 2002 3:00 am
Posts: 516
Location: Virginia
Unread post Re: TWX Technical Discussion (and decompiling info)
More fun with decompiling.

I've run into a few scenarios where I saw slightly unexpected results for complex if/else and (and/or) statements, and I think it's mostly due to a misunderstanding of how the compiler handles these -- in some cases, the original scripts were not working correctly after testing them.

TWXC processes all compound statements from left to right. And while it ignores parentheses in many cases, it does pay attention to them when determining the order in which it processes statements.

Here are some examples.

[[I apologize for the screwed up tabs, I am not sure how to fix that in this forum]]

This statement: if (($P = "ore") and $BF) and $BO
Compiles to: if ((($P = "ore") and $BF) and $BO)

Here's how it is parsed (into bytecode):

Code:
ISEQUAL $$3 $P "ore"        $$3: ($P = "ore")
SETVAR $$2 $$3                $$2: ($P = "ore")
AND $$2 $BF                     $$2: (($P = "ore") and $BF)
SETVAR $$1 $$2                $$1: ($P = "ore") and $BF
AND $$1 $BO                     $$1: ((($P = "ore") and $BF) and $BO)


The reason is, the compiler treats each "and" as a separate branched part of the statement, and handles them in order from left to right.

You'll notice the $$1 etc variables. This is how the compiler breaks down any statement that has "and/or" macros in it.

The original code:

if (($P = "ore") and $BF) and $BO
halt
end

The bytecode, once compiled by TWXC:

Code:
ISEQUAL $$3 $P "ore"
SETVAR $$2 $$3
AND $$2 $BF
SETVAR $$1 $$2
AND $$1 $BO
BRANCH $$1 :5
HALT
:5


Okay, let's look at another example.

This statement: if ($P = "ore") and $BF and $BO
Compiles to: if (($P = "ore") and ($BF and $BO))

Code:
ISEQUAL $$2 $P      $$2: ($P = "ore")
SETVAR $$5 $BF      $$5: $BF
AND $$5 $BO      $$5: ($BF and $BO)
SETVAR $$1 $$2      $$1: ($P = "ore")
AND $$1 $$5      $$5: (($P = "ore") and ($BF and $BO))


Again, the compiler is processing from left to right. So we end up with:

if (($P = "ore") and $BF) and $BO compiled to if ((($P = "ore") and $BF) and $BO)
if ($P = "ore") and $BF and $BO compiled to if (($P = "ore") and ($BF and $BO))

Obviously, the result of this is important. Depending on what you want your .ts code to do, one will work and one will not.

Here's a more complicated example, and one I actually had to fix recently.

This statement:

if (($P = "ore") and $BF) or (($P = "org") and $BO) or (($P = "equ") and $BE) or (($P = "") and ($PC < 8))

Compiles like this:

Code:
ISEQUAL $$3 $P "ore"   $$3:   ($P = "ore")
SETVAR $$2 $$3      $$2:   ($P = "ore")
AND $$2 $BF      $$2:   (($P = "ore") and $BF)
ISEQUAL $$9 $P "org"   $$9:   ($P = "org")
SETVAR $$8 $$9      $$8:   ($P = "org")
AND $$8 $BO      $$8:   (($P = "org") and $BO)
ISEQUAL $$15 $P "equ"   $$15:   ($P = "equ")
SETVAR $$14 $$15   $$14:   ($P = "equ")
AND $$14 $BE      $$14:   (($P = "equ") and $BE)
ISEQUAL $$20 $P ""   $$20:   ($P = "")
ISLESSER $$23 $PC 8   $$23:   ($PC < 8)
SETVAR $$19 $$20   $$19:   ($P = "")
AND $$19 $$23      $$19:   (($P = "") and ($PC < 8))
SETVAR $$13 $$14   $$13:   (($P = "equ") and $BE)
SETVAR $$13 $$14   $$13:   ($P = "equ")
OR $$13 $$19      $$13:   ($P = "equ") and $BE) OR (($P = "") and ($PC < 8)
SETVAR $$7 $$8      $$7:   (($P = "org") and $BO)
OR $$7 $$13      $$7:   (($P = "org") and $BO) OR (($P = "equ") and $BE) OR (($P = "") and ($PC < 8)
SETVAR $$1 $$2      $$1:   (($P = "ore") and $BF)
OR $$1 $$7      $$1:   (($P = "ore") and $BF) OR (($P = "org") and $BO) OR (($P = "equ") and $BE) OR (($P = "") and ($PC < 8)


Meanwhile, this version:

if ($P = "ore") and $BF or ($P = "org") and $BO or ($P = "equ") and $BE or ($P = "") and ($PC < 8))

Compiles to:

Code:
ISEQUAL $$2 $$P "ore"   $$2:   ($P = "ore")
ISEQUAL $$8 $P "org"   $$8:   ($P = "org")
ISEQUAL $$14 $P "equ"   $$14:   ($P = "equ")
ISEQUAL $$20 $P ""   $$20:   ($P = "")
ISLESSER $$23 $PC 8   $$23:   ($PC = 8)
SETVAR $$19 $$20   $$19:   ($P = "")
AND $$19 $$23      $$19:   (($P = "") AND ($PC = 8))
SETVAR $17 $BE      $$17:   $BE
OR $$17 $$19      $$19:   $BE OR (($P = "") AND ($PC = 8))
SETVAR $$13 $$14   $$13:   ($P = "equ")
AND $$13 $$17      $$13:   (($P = "equ") AND ($BE OR (($P = "") AND ($PC = 8)))
SETVAR $$11 $BO      $$11:   $BO
OR $$11 $$13      $$11:   $BO OR (($P = "equ") AND ($BE OR (($P = "") AND ($PC = 8)))
SETVAR $$7 $$8      $$7:   ($P = "ORG")
AND $$7 $$11      $$7:   (($P = "ORG") AND ($BO OR (($P = "equ") AND ($BE OR (($P = "") AND ($PC = 8))))
SETVAR $$5 $BF      $$5:   $BF
OR $$5 $$7      $$5:   $BF OR (($P = "ORG") AND ($BO OR (($P = "equ") AND ($BE OR (($P = "") AND ($PC = 8))))
SETVAR $$1 $$2      $$1:   ($P = "ore")
AND $$1 $$5      $$1:   (($P = "ore") AND ($BF OR (($P = "ORG") AND ($BO OR (($P = "equ") AND ($BE OR (($P = "") AND ($PC = 8)))))


So again, we end up with two very different results, one of which is probably not the one the script writer intended :)

Hope this was useful.

_________________
TOURNAMENT WINNER: ICE 2017 - ICE 2019 - SUMMER SPLASH 2019 - XMAS TOURNEY 2019


Mon Oct 07, 2019 1:43 pm
Profile ICQ
Ambassador
User avatar

Joined: Wed Apr 20, 2011 1:19 pm
Posts: 2559
Location: Oklahoma City, OK 73170 US
Unread post Re: TWX Technical Discussion (and decompiling info)
How hard would it be to add Switch to TWX?

Code:
Switch (A)
    case 1:
        do this
        break
    case 2:
       do that
       break
    default:
       kill something
       break
end


is a lot prettier than:

Code:
IF (A = 1)
        do this
elseif (A= 2)
       do that
else
       kill something
end

_________________
Regards,
Micro

Website: http://www.microblaster.net
TWGS2.20b/TW3.34: telnet://twgs.microblaster.net:2002

ICQ is Dead Jim! Join us on Discord:
https://discord.gg/zvEbArscMN


Mon Oct 07, 2019 4:13 pm
Profile ICQ YIM WWW
Lieutenant
User avatar

Joined: Tue Dec 17, 2002 3:00 am
Posts: 516
Location: Virginia
Unread post Re: TWX Technical Discussion (and decompiling info)
Micro wrote:
How hard would it be to add Switch to TWX?


You tell me, I'm not a Delphi programmer, nor can I even compile Delphi code :)

It shouldn't be hard, and would be a nice language extension, though it is debatable whether it is worth adding it now or waiting for the c# port. I am making progress on that.

_________________
TOURNAMENT WINNER: ICE 2017 - ICE 2019 - SUMMER SPLASH 2019 - XMAS TOURNEY 2019


Mon Oct 07, 2019 5:07 pm
Profile ICQ
Lieutenant J.G.

Joined: Mon Dec 01, 2014 5:39 pm
Posts: 440
Unread post Re: TWX Technical Discussion (and decompiling info)
Shadow wrote:
Micro wrote:
How hard would it be to add Switch to TWX?


You tell me, I'm not a Delphi programmer, nor can I even compile Delphi code :)

It shouldn't be hard, and would be a nice language extension, though it is debatable whether it is worth adding it now or waiting for the c# port. I am making progress on that.


Are you taking feature requests? :)

Are you planning on making the source public once complete? Might be able to get some help to add features, including from me.

Some ideas I've had
* TWX GUI Front End (on top of the awesome work Micro is doing now)
- import game settings from Micro's site - so you can have one click database add
- Database management - Id love a plain Jane list of databases so I could see it in a long table vs a drop down. Ability to delete everything linked to a DB easier (Without essentially loading it first), add a note etc "db for testing X Y Z" - I have 5-6 working TWX directories with dozens of games.. it's just easier to start a new directory and delete an old directory at the moment. This is probably more of an issue for when I'm doing testing on things like mombot logins routines/game starts.. but still.

TWX Scripting - I can think of lots of ideas to make it more usable but no idea how hard the to change the parser will be to modify. Imagine being able to parse var's to a function in a more traditional form

i.e.
if (goSubParam ($param1, $param2) = $subResult)
or in code it'd look like
if (goSubParam (1, 2) = TRUE)

which looks like this now
Code:
setVar $param1 1
setVar $param2 2
setVar $paramResult 0
goSub :goSubParam

if ($paramResult = True)



oh I can dream..


Mon Oct 07, 2019 7:56 pm
Profile
Ambassador
User avatar

Joined: Wed Apr 20, 2011 1:19 pm
Posts: 2559
Location: Oklahoma City, OK 73170 US
Unread post Re: TWX Technical Discussion (and decompiling info)
The source has been on GitHub for a long time:
https://github.com/MicroBlaster/TWXProxy

Code:
if (goSubParam ($param1, $param2) = $subResult)


Since TWX commands don't have a concept of returning a value, this isn't possible.

Many of your other ideas are possible.

_________________
Regards,
Micro

Website: http://www.microblaster.net
TWGS2.20b/TW3.34: telnet://twgs.microblaster.net:2002

ICQ is Dead Jim! Join us on Discord:
https://discord.gg/zvEbArscMN


Tue Oct 08, 2019 6:55 am
Profile ICQ YIM WWW
Lieutenant
User avatar

Joined: Tue Dec 17, 2002 3:00 am
Posts: 516
Location: Virginia
Unread post Re: TWX Technical Discussion (and decompiling info)
Micro wrote:
The source has been on GitHub for a long time:
https://github.com/MicroBlaster/TWXProxy

Code:
if (goSubParam ($param1, $param2) = $subResult)


Since TWX commands don't have a concept of returning a value, this isn't possible.

Many of your other ideas are possible.


Isn't possible today, you mean. Anything is possible. :)

There is no current concept of returning a value, that is absolutely correct, but it is not very different from how scripts work today. They are processed in a linear order, yes, but there are gotos and gosubs and branches. Think of what he's asking for as a branch that goes to one place if the "return value" is x or another if it is y. It wouldn't be that hard.

_________________
TOURNAMENT WINNER: ICE 2017 - ICE 2019 - SUMMER SPLASH 2019 - XMAS TOURNEY 2019


Tue Oct 08, 2019 7:39 am
Profile ICQ
Display posts from previous:  Sort by  
Reply to topic   [ 22 posts ]  Go to page 1, 2  Next

Who is online

Users browsing this forum: No registered users and 25 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by STSoftware.