View unanswered posts | View active topics It is currently Thu Oct 23, 2014 4:56 pm



Reply to topic  [ 27 posts ]  Go to page 1, 2  Next
 ZTM Scripts 
Author Message
Ambassador
User avatar

Joined: Mon Feb 09, 2004 3:00 am
Posts: 3138
Location: Kansas
Unread post 
Just needing some feedback on a couple of ZTM scripts I am writing. The first is a rewrite of proztm_06 (proZTM_07) and the second is ProZTM_Speed.....

ProZTM_07 - found 52,525 of 52,595 (observer account) 99.86% of the warps in a 20k sub zero edit. It required 6 hours and 34 minutes to complete, although the first 45,970 were found in 1 hour and 8 minutes.

ProZTM_Speed found 51,222 of 52,595 warps or 97.4% in 4 hours 37 minutes (I think it may have been 3 hours and 37 minutes, but I will have to verify that - was half-asleep when I closed the window).

Testng in a 500 sector (so it is quick to complete) bang showed 11 minutes 10 seconds vs 5 minutes 43 seconds. (ProZTM_06, if you have used it, required 12 minutes, 48 seconds.)

Both scripts use the same method for the first pass so the 45,970 warps (87.4%) were found in the same time.

Where I need some feedback, is the loss in accuracy of about 2.5% offset by the approximately 1/2 time it takes to create a ZTM?

I have not released either the updated script or the "speed" version yet, and I am wanting to gauge any interest in the "speed" version.

One thing that will be added to the ztm's that I am writing is a auto-relog and resume. It is kind of irritating to start a ztm and come back 3 hours later only to find that I have lost connection and have to $st to find the trigger to restart a script at.

_________________
               / Promethius / Enigma / Wolfen /

"A man who has no skills can be taught, a man who has no honor has nothing."


Fri May 12, 2006 6:12 pm
Profile ICQ
Captain
User avatar

Joined: Sat Jun 18, 2005 2:00 am
Posts: 2214
Location: USA
Unread post 
Would using something like in the routine like:

IF CONNECTED 0
GoTo :SaveMyCurrentZTMPosition&StandbyForReLogon&Continue
END

Or is TWX unable to detect that type of info, while ZTMing? I have been wondering why is it so difficult for a ZTM to reach 100%? Has anybody actually tested being able to visit all 20,000 sectors in various Bangs or is there a flaw in the universe design or possibily in the CIM itself?

_________________
Your reliance upon subjective IRM's, subjugates you through utter omission, obfuscation, and distortion of fact!
Don't mess with me, I will 26 U.S.C. § 7212(a) your IRS!


Sat May 13, 2006 12:56 am
Profile ICQ WWW
Veteran Op
User avatar

Joined: Thu Jun 02, 2005 2:00 am
Posts: 5558
Location: USA
Unread post 
Hey pro... do you burst 5, 10, or what, plots at a time? Been wrestling with that a bit... more means faster on high lag but more plots total, less means fewer plots but slower on high pings. How many plots do you average on the first pass thru a 20k map?

Personally, I've always liked the multiple pass concept so I can apply what I need for the situation.

A 2-pass plus 1-way checks will suffice in an unlim or high turns game. Takes between 2.5 and 3 hours, 50000 out of 57000 warps last time I checked. Good if I plan to follow it up with a lot of gridding to round things out. I could add another pass and probably get quite a few more warps for not a lot of time, but since I really only need a few adjs and the DEs to aim for there's no point in spending grid time doing a map.

Usually tho I'll wait till I have a few hours and run a 6-pass, 1-ways and datamining. Whole thing can take up to 8 hours, but it's accurate to within 30 warps and produces a lot of extra stuff to make life easier. I'm normally afk or asleep anyway when this is going on so the extra time is moot.

If I need accuracy in a hurry I just do a 4-pass plus 1-ways. It misses up to 100 warps but can complete in about 5 hours. Since I'm usually not watching the ztm tho it doesn't normally come up unless someone comes on and starts hitting stuff they shouldn't.

Rexx, most ZTMs automatically keep track of the last plots done and the current pass and can be re-activated whenever they need to be. Just kill the script and restart, it'll pick right back up.

And yes we've all probably visited 100% of the universe. I do it all the time. The ZTM can't really reach 100% (atleast not in all cases) because of the complexities of the universe and the way course plots work. It's possible to get 99.999999% but the higher you go the more time it takes. Why spend 11 hours getting 99.999999999999% when you can spend 8 and get 99.9%? A whopping 20 to 30 out of 55k warps rarely causes problems, even in the most sophisticated data mining setups.

_________________
May the unholy fires of corbomite ignite deep within the depths of your soul...

1. TWGS server @ twgs.navhaz.com
2. The NavHaz Junction - Tradewars 2002 Scripts, Resources and Downloads
3. Open IRC chat @ irc.freenode.net:6667 #twchan
4. Parrothead wrote: Jesus wouldn't Subspace Crawl.

*** SG memorial donations via paypal to: dpocky68@booinc.com
Image


Sat May 13, 2006 3:04 am
Profile ICQ WWW
Ambassador
User avatar

Joined: Mon Feb 09, 2004 3:00 am
Posts: 3138
Location: Kansas
Unread post 
quote:Originally posted by Singularity

Hey pro... do you burst 5, 10, or what, plots at a time? Been wrestling with that a bit... more means faster on high lag but more plots total, less means fewer plots but slower on high pings. How many plots do you average on the first pass thru a 20k map?

Personally, I've always liked the multiple pass concept so I can apply what I need for the situation.

A 2-pass plus 1-way checks will suffice in an unlim or high turns game. Takes between 2.5 and 3 hours, 50000 out of 57000 warps last time I checked. Good if I plan to follow it up with a lot of gridding to round things out. I could add another pass and probably get quite a few more warps for not a lot of time, but since I really only need a few adjs and the DEs to aim for there's no point in spending grid time doing a map.

Usually tho I'll wait till I have a few hours and run a 6-pass, 1-ways and datamining. Whole thing can take up to 8 hours, but it's accurate to within 30 warps and produces a lot of extra stuff to make life easier. I'm normally afk or asleep anyway when this is going on so the extra time is moot.



I haven't checked the actual number of plots that the script runs - might be interesting to add a counter for that though. The edit I am testing on has a max distance of 45 warps. Longer max warps should speed things up I would think.

The bursts are single plot with no delay before a second plot. I have two passes, but each pass contains multiple plot types. I also have a verify of each sector from the TWX database after the first pass to insure I have data on each sector at that point. The "speed" version does not use avoids and that is why it is considerably faster, but not as accurate, at my standard ZTM. 97.4% accuracy is really not as good as it sounds when one considers the number of missed warps. I am going to try boosting the accuracy to the 98.5 to 99% accuracy range with hopefully only a 20-30 minute penalty.

I ran a second test to just verify time and the "speed" version completed the 20k sector ztm in 3 hours and 37 minutes (second actually a bit faster). The tests were ran after I put a player in sector 1 so the only thing the database sees at startup are the sectors surrounding 1.

_________________
               / Promethius / Enigma / Wolfen /

"A man who has no skills can be taught, a man who has no honor has nothing."


Sat May 13, 2006 5:51 am
Profile ICQ
Veteran Op
User avatar

Joined: Thu Jun 02, 2005 2:00 am
Posts: 5558
Location: USA
Unread post 
IIRC, the DB is updated on each waitFor/waitOn/trigger. So if you send a bunch of plots w/o a waitOn I wonder if it updates the warp count for each sector during the run. If not, then it would be doing far more plots than it needs to if you use Rev's warp-based method. Be interesting to get a plot count, it seems like a good benchmark.

How are you getting 97% accuracy w/o avoids? How can you force a course to travel a different door without the use of avoids? Be interesting to see that in action, how are selecting your start/dest sectors for each plot? Standard X & SECTORS-X, X & X+1... or something more unique?

97% means 2k missed warps in an avg 20k uni, which isn't great but is enough to grid with if you follow up with a 1-ways check. Not sure if an extra 1 or 2% is worth spending any time at all, tho. For me it's either "just enough warp data" or "all the warp data I can use," there really isn't a lot of middle ground. Either I'm in a hurry or I'm not.

_________________
May the unholy fires of corbomite ignite deep within the depths of your soul...

1. TWGS server @ twgs.navhaz.com
2. The NavHaz Junction - Tradewars 2002 Scripts, Resources and Downloads
3. Open IRC chat @ irc.freenode.net:6667 #twchan
4. Parrothead wrote: Jesus wouldn't Subspace Crawl.

*** SG memorial donations via paypal to: dpocky68@booinc.com
Image


Sat May 13, 2006 6:37 am
Profile ICQ WWW
Ambassador
User avatar

Joined: Mon Feb 09, 2004 3:00 am
Posts: 3138
Location: Kansas
Unread post 
I am running a test on the speed ztm now. I will post it on grimy if all works out well as a beta version, and maybe in source format. The addition to try and grab a better percentage uses avoids for a 3rd pass that works the deadends. This will slow the process down and unless I gain a decent number of warps found, I will remove it.

The number of plots in a 20k sector will get to 97% provided the sectors are selected the way I have done it (may be a better method out there). The first pass guarantees I have data on each sector - well, after the verify runs anyway. The start/dest sectors for the next pass are determined based on the data I have for those sectors from the first plot.

I am surprised that this method is as accurate as it seems to be. I figured the speed increase would be substantial, but the initial accuracy is unexpected.

_________________
               / Promethius / Enigma / Wolfen /

"A man who has no skills can be taught, a man who has no honor has nothing."


Sat May 13, 2006 8:04 am
Profile ICQ
Veteran Op
User avatar

Joined: Thu Jun 02, 2005 2:00 am
Posts: 5558
Location: USA
Unread post 
Ok. I've been doing a low-high pair. Initial pass runs thru X and SECTORS - X. From there I only plot to sectors w/ matching warp counts. For instance on the second pass where I clarify out DEs I look for any sector w/ 1 warp in to plot to. The hope being, of course, that it might find another warp in. I have considered interspersing a 1-way check in between every or every-other pass but I haven't done that yet... let alone testing it's effectiveness.

This continues from pass 0 to pass 5, each # being the number of warps out I check. Pretty much the same approach CK uses. Always looking for ways to improve the results.

When you get the .cts ready I'll test it out if you want.

_________________
May the unholy fires of corbomite ignite deep within the depths of your soul...

1. TWGS server @ twgs.navhaz.com
2. The NavHaz Junction - Tradewars 2002 Scripts, Resources and Downloads
3. Open IRC chat @ irc.freenode.net:6667 #twchan
4. Parrothead wrote: Jesus wouldn't Subspace Crawl.

*** SG memorial donations via paypal to: dpocky68@booinc.com
Image


Sat May 13, 2006 10:24 am
Profile ICQ WWW
Lieutenant

Joined: Sat Nov 06, 2004 3:00 am
Posts: 600
Location: USA
Unread post 
u talk about the DB being updated, I thought twx logs everything... I still use rev's ztm for zoc and twx seems to catch it all. Most warps in 20k seem to be between 52k to 53k range, but I sure it misses some too.

_________________
Its not the Kill, Its the Thrill of the Chase
S: Min: 297 Max: 437 Average: 410 -- Just here to attack aliens
S: High/Low Removed   Average: 421
          


Sat May 13, 2006 3:04 pm
Profile ICQ WWW
Veteran Op
User avatar

Joined: Thu Jun 02, 2005 2:00 am
Posts: 5558
Location: USA
Unread post 
IIRC it's not that doesn't log it, it's when it chooses to do that logging...

_________________
May the unholy fires of corbomite ignite deep within the depths of your soul...

1. TWGS server @ twgs.navhaz.com
2. The NavHaz Junction - Tradewars 2002 Scripts, Resources and Downloads
3. Open IRC chat @ irc.freenode.net:6667 #twchan
4. Parrothead wrote: Jesus wouldn't Subspace Crawl.

*** SG memorial donations via paypal to: dpocky68@booinc.com
Image


Sat May 13, 2006 6:00 pm
Profile ICQ WWW
Commander
User avatar

Joined: Tue Oct 07, 2003 2:00 am
Posts: 1047
Location: Augusta, GA
Unread post 
Actually, the DB in TWX is updated immediately. A packet is broken down into lines, and each line is parsed, and relevant data is added to the DB immediately, even before triggers can activate.

I think what you are referring to Sing is the fact that TWX can't process new incoming lines if it is stuck processing something else, hence the need to be paused.

Also, I do my destination selection exactly that same way Sing. I figure why plot to a sector with 6 known warps in.

Promethius, you can plot using avoids without delays, so long as you macro the setting and clearing of these avoids, the time increase should be negligible. The only part where I think it's somewhat necessary to delay is on the one-way test, which is usually a small percentage of the overall ZTM time.

And for the record, I'm confident that a 100% accurate ZTM can be done without significant time increase as compared to a 6-pass routine. Difficult, but possible.

+EP+

_________________
Claim to Fame: only guy to ever crack the TW haggle algorithm, and fig/shield/hold price formula.


Mon May 15, 2006 4:24 am
Profile WWW
Veteran Op
User avatar

Joined: Thu Jun 02, 2005 2:00 am
Posts: 5558
Location: USA
Unread post 
Why is it neccessary to delay on the one-way test? I don't delay w/ anything out of the ordinary (cache records out, wait for a middle one to return... you've seen the code) and it seems to work fine. Unless I'm missing something.

How can a 100% accurate ZTM be done w/o a significant time increase? I've been looking at the list of missed warps and while I've not spent a great deal of time thinking about them, they all see to be in bubbles where the first avoid basically blocks the 2nd or subsequent passes.

Only way I could think of doing it would be to use a first pass to try and track distance and plot to closer sectors. That sounds ok at first, but then how can I know the difference between a real DE and 2 linked sectors off a bubble gate beforehand so I don't plot short courses on all sectors (increasing the overall plot requirement and therefore increasing time)?

Prom is using an X & SECTORS-X followed by a backplot. I added that to my first pass and for the added cost of about 10 minutes it made my 2nd and 3rd passes quite a bit more accurate. Results evened out at about the 4th pass.

_________________
May the unholy fires of corbomite ignite deep within the depths of your soul...

1. TWGS server @ twgs.navhaz.com
2. The NavHaz Junction - Tradewars 2002 Scripts, Resources and Downloads
3. Open IRC chat @ irc.freenode.net:6667 #twchan
4. Parrothead wrote: Jesus wouldn't Subspace Crawl.

*** SG memorial donations via paypal to: dpocky68@booinc.com
Image


Mon May 15, 2006 7:46 am
Profile ICQ WWW
Lieutenant Commander
User avatar

Joined: Thu Mar 21, 2002 3:00 am
Posts: 890
Location: USA
Unread post 
EP and I noodled it out on paper, and I think we decided the amount of code needed wasn't worth the effort to find the occasional stray warps. The current ZTM we use gets 100% of the warps in 90% of the games I've used it in. In fact, it used to be annoying, because I would loose warps when I imported other's TWX DB's. I had to remember to save my warps before importing others. EP's fixed that in the latest version.

EP probably has a better memory of how we figured it out, as he actually bothered to take notes. (I stopped when I realized how much of a pain it would be.) [:D]

_________________
http://tw-cabal.navhaz.com - THE TW info site

Man, I gotta quit showing up here...next thing you know i'll get dragged back in.


Mon May 15, 2006 4:50 pm
Profile WWW
Ambassador
User avatar

Joined: Mon Feb 09, 2004 3:00 am
Posts: 3138
Location: Kansas
Unread post 
EP, I have been bursting my initial avoids and plots at one time; however since I grab additional sectors to avoid and replot, it takes a second burst - unless the first burst resulted in clearing avoids. This still seems slower than plotting w/o avoids.

Singularity is correct on how I am plotting on the "X, SECTORS-X" method. The first sector is checked for warpcount and if 0 selected for plot, the second sector is checked for warpincount and if 0 selected for plot. A backplot is used if the start sector does not have a warpin or the end sector does not have a warp out - come to think of it for the backplot, that is probably always the case on the first pass.

I get a warpout and warpin for every sector on the first pass, and a verify section makes sure nothing was missed. The first pass is about 85% accurate and takes 1 hour 10 minutes to 1 hour 20 minutes to complete.

In the "speed" version I use plots of like warp sectors - plot 1 warp to 1 warp, until I get to sector plots with 3 or 4 warps - 3 seems almost as accurate as 4 in a 20k sector game. When this is complete, another plot of 1 warp to 1 warp sectors is ran using avoids. A 20K ZTM using this method is about 95 - 97.5% accurate if I remember right.

In the proZTM_07 version, I use avoids for the second pass - bursting known avoids and the plot and if additional sectors are found, sending an avoid the the new sector and plot in a burst. This seems to be about 99.8 to 99.9% accurate - missing about 70 warps in a 20k sector test and takes about 6 to 6.5 hours to complete.

I've posted the source to the "speed" version on grimy and it includes the code for finding 7warps-in (based on EP's 7warpsin.ts), dead end sectors (which should be rewritten since there is a better way to do it and probably more accurate), and 6 sector warps.

_________________
               / Promethius / Enigma / Wolfen /

"A man who has no skills can be taught, a man who has no honor has nothing."


Mon May 15, 2006 5:26 pm
Profile ICQ
Commander
User avatar

Joined: Tue Oct 07, 2003 2:00 am
Posts: 1047
Location: Augusta, GA
Unread post 
Yeah, I hated that as well. You spend the time doing a ZTM, then import your corpies' TWX file, only to lose warps due to their lesser ZTM. So now, in TWX 2.04, if the imported sector has less warps than the copy in your DB, it won't overwrite them. Yea!

That approach regarding the back plot isn't beneficial over Rev's basic approach. It all boils down to .5 sec per plot, and there isn't a more efficient method in regard to {warps discovered/time} than the plot by pass approach. If you take the standard approach and the back-plot approach, I'll wager that in an equal duration of time, the standard will have a higher number of warps. Instead, you measured by pass count. Keep your efficiency measurement in warps discovered per qty time.

This isn't to say that Rev's plot-by-pass method can't be improved upon. I can think of a number of improvements, all very difficult to implement, and little return on investment. I spoke with Rev in depth, and he was convinced (after tons of research) that a 3 pass approach yielded the best accuracy vs time ratio. So I say, either do 3 passes, or do your best to get 100% no matter how long it takes. Remember though that the one-way check is always necessary.

Sing, you are right on about the missing warps. Keep thinking about it and hit me on ICQ to discuss it. I'll give you hint, it requires about 1 plot per gate.

+EP+

_________________
Claim to Fame: only guy to ever crack the TW haggle algorithm, and fig/shield/hold price formula.


Mon May 15, 2006 5:40 pm
Profile WWW
Ambassador
User avatar

Joined: Mon Feb 09, 2004 3:00 am
Posts: 3138
Location: Kansas
Unread post 
I will try the script w/o the back plots and see how that works out. One would "think" that a ztm should be fairly simple - for each sector simply plot and set avoids until there are no more warps out, but it just isn't that easy. Logically it would appear that if all warps out of each sector are plotted using avoids, there would be nothing left to work as that should also give all warps in. (For anyone wondering, yes I tried this approach in my first try at a ZTM and failed miserably).

ZTM mapping reflects playing TradeWars pretty well - what appears simple, isn't necessarily so.

_________________
               / Promethius / Enigma / Wolfen /

"A man who has no skills can be taught, a man who has no honor has nothing."


Mon May 15, 2006 6:29 pm
Profile ICQ
Display posts from previous:  Sort by  
Reply to topic   [ 27 posts ]  Go to page 1, 2  Next

Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by STSoftware.