Get The Latest Compile Tools
Cagey
Ex-Unknown Worlds Programmer Join Date: 2002-11-15 Member: 8829Members, Retired Developer, NS1 Playtester, Constellation
<div class="IPBDescription">Updated ZHLT: Information & Downloads</div> <b>Latest download: </b> <a href='http://xp-cagey.com/downloads/zhlt_1.7p15.zip' target='_blank'>ZHLT 1.7p15</a>
<b>Earlier versions: </b> <a href='http://xp-cagey.com/downloads/zhlt_1.7p13.zip' target='_blank'>ZHLT 1.7p13</a>
<b>Cagey's full source code: </b> <a href='http://www.xp-cagey.com/downloads/zhlt_p15.5_source.zip' target='_blank'>ZHLT 17p15.5 source</a> <span style='color:red'>Note: when using the source code, you should refer to my notes on page 33 of this topic</span>.
For those who aren't aware, this is an update of Merl's 1.7 Custom Build of Zoner's tools. All of the features that Merl and Zoner added are still available in this distribution.
<span style='color:red'>See the red notes below for important information on the latest build.</span>
<!--QuoteBegin-^Requiem^+Jan 26 2003, 08:26 PM--></div><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (^Requiem^ @ Jan 26 2003, 08:26 PM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin-->I see that a fix for a larger amount of maxplanes might be heading our way in the future, but I'm trying to discern if there is anything available to me right NOW to allow me to build a more compilcated level.<!--QuoteEnd--></td></tr></table><div class='postcolor'><!--QuoteEEnd-->
When you posted there wasn't, but I've gotten Merl's permission to post modified copies of version 1.7 with MAX_MAP_PLANES set to 65535. Replace your build tools with these versions and you'll be able to compile bsps with over 32767 planes. Then use my optimizer (included in the zip files) to make them safe for the game again. I'm putting them into two zip files since they're over the 122880 byte max file limit in a single download.
To use my optimizer, you can drag and drop your bsp file over it. The program overwrites the original file, so make a backup if you want to preserve the unoptimized copy. **Note -- since this program shrinks the map, it will checksum differently -- if a server is running the unoptimized copy and you have the optimized, it might not let you connect **.
Once the map is optimized, HLCSG -onlyents and HLRAD will break (make the optimization the final step when you're compiling), but it will run fine in the game.
<span style='color:gray'>EDIT: If you get a 'file not found' error when running the tools, look at Ollj's post below, he has a solution to the problem. You can get the DLL he talks about <a href='http://www.cyscape.com/download/msvcr70.dll' target='_blank'>here</a>. If you still can't get the tools to run, two people have reported needing a second DLL you can grab from <a href='http://www.dll-files.com/dllindex/dll-files.shtml?msvcp70' target='_blank'>here</a>, but others seem to be OK without it.</span>
<span style='color:gray'>EDIT 2/4/03: updated opt_plns to version 1.2, fixing the crash on bsp file not found and fixing a rare case of corrupted data for the first plane on the map.</span>
<span style='color:gray'>EDIT 2/5/03: updated opt_plns (in first download) again to version 1.3, fixing a crash bug above 32K planes and adding an option (-nopause) to skip the keypress prompt at the end of execution. The updated opt_plns is available as a separate program on the 4th page of this thread. Unless another bug is found, this should be the last update for a while.</span>
<span style='color:gray'>EDIT 2/8/03: well that was dumb of me <!--emo&???--><img src='http://www.unknownworlds.com/forums/html//emoticons/confused.gif' border='0' style='vertical-align:middle' alt='confused.gif' /><!--endemo-->. I posted the opt_plns zip instead of the modified tools 1 zip that includes the new opt_plns here -- thanks to [builder]hicks for helping me catch the problem.</span>
<span style='color:gray'>EDIT 2/11/03: updated opt_plns again to version 1.4, removing internal limits on texdata, visdata, and lightdata. Also added a -logfile <filename> switch and made input filename parsing more forgiving.</span>
<span style='color:gray'>EDIT 2/21/03: updated the tools to release 1.7p3, which speeds up HLCSG and fixes several clipping bugs as detailed <a href='http://xp-cagey.com/articles/clip_hull' target='_blank'>here</a>. There is a thread talking about the HLCSG changes <a href='http://www.unknownworlds.com/forums/index.php?act=ST&f=4&t=24137&st=0&' target='_blank'>here</a>. You'll want to download the second attachment again if you have grabbed these files before.</span>
<span style='color:gray'>EDIT 3/13/03: updated the tools to release 1.7p4, which contains a threadlock bugfix, code that is more friendly to concave brushes, and a new -cliptype normalized option. More details are available <a href='http://www.unknownworlds.com/forums/index.php?act=ST&f=4&t=24137&st=0&' target='_blank'>here</a>.</span>
<span style='color:gray'>EDIT 4/09/03: updated the tools to release 1.7p5, which added some new features and made some optimizations. More details are available <a href='http://www.unknownworlds.com/forums/index.php?act=ST&f=4&t=21248&st=105' target='_blank'>here</a>.</span>
<span style='color:gray'>EDIT 4/10/03: the 1.7p5 zipfiles contained the wrong version of the tools yesterday--my apologies to the 11 people who downloaded those files -- the links are now correct.</span>
<span style='color:gray'>EDIT 4/10/03: quick turnaround to version 1.7p6 to fix a show stopping bug related to the new BEVEL texture code in HLCSG.</span>
<span style='color:gray'>EDIT 4/13/03: released 1.7p7 added texlight support back into the tools and fixed problem of "too many direct lightstyle" message spam (you now will see the message once instead of every time the problem occurs). This is a stable release.</span>
<span style='color:gray'>EDIT 4/17/03: some people have had problems with the new lighting code, so I've added a switch, -oldmath, to RAD that allows using the older basic math functions. The new release has been tagged 1.7p8.</span>
<span style='color:gray'>EDIT 5/4/03: the -oldmath tag has been deprecated since I found the source of the problem. Also added support for the SDK 2.2 hull file format. Released as 1.7p9.</span>
<span style='color:gray'>EDIT 5/8/03: fixed a bug in opaque entity handling, released as 1.7p10.</span>
<span style='color:gray'>EDIT 8/15/03: maintenance release -- throw fatal error on buffer overflows that used to be silent, added location information to "too many lightsamples" error, using -verbose on HLRAD now displays ALL of those errors (there can be thousands in a single map). Added "-lightdata #" command so that people can add extra lightdata to their maps--doing this, like adding texdata, will require you to test your map on low-end machines to make sure that people can play your map. Released as 1.7p11</span>
<span style='color:gray'>EDIT 8/22/03: Rolling back publicly available release to p10 due to lighting bug report. Will release p11 again once the bug has been resolved.</span>
<span style='color:gray'>EDIT 10/3/03: Resolved the issue in p11, commented out spamming would-be developer message from HLRAD, reset default lightdata limit back to 6MB. Thanks to Kage and WolfWings for testing the release cantidates.</span>
<span style='color:gray'>EDIT 2/3/04: p13: Fixed bug in creation of objects without clip hulls, fixed bug in info_compile_parameters entity handler, changed usage of zhlt_noclip and zhlt_invisible to allow inclusion of those properties in .fgd files. Added Hullu's latest lighting enhancements.</span>
<span style='color:gray'>EDIT 6/1/04: p14: Fixed bug that caused errors building under default steam installs. Added switch to turn off light bounces for dynamic lights. Built optimizer into HLBSP. Raised internal plane limit to 256K. Added Reve's overhauled documentation.</span>
<span style='color:gray'>EDIT 6/3/04: p15: Fixed filehandler crash bug. Fixed face index crash bug. Added switch to turn off plane optimization. Fixed SolidBSP display times. Fixed bug with opaque entity lighting. Changed chart output to display planes percentage based on Half-Life plane limit.</span>
<span style='color:red'>EDIT 8/15/04: source release: I've released my latest source a few days ago since I've been hired by a game company and won't be working on ZHLT any more. Notes about the release are in the original posting on page 33 of this topic; I forgot to post a first-page update when I uploaded the source, but it's here now.</span>
Enjoy! <!--emo&:)--><img src='http://www.unknownworlds.com/forums/html//emoticons/smile.gif' border='0' style='vertical-align:middle' alt='smile.gif' /><!--endemo-->
<b>Latest download: </b> <a href='http://xp-cagey.com/downloads/zhlt_1.7p15.zip' target='_blank'>ZHLT 1.7p15</a>
<b>Earlier versions: </b> <a href='http://xp-cagey.com/downloads/zhlt_1.7p13.zip' target='_blank'>ZHLT 1.7p13</a>
<b>Cagey's full source code: </b> <a href='http://www.xp-cagey.com/downloads/zhlt_p15.5_source.zip' target='_blank'>ZHLT 17p15.5 source</a> <span style='color:red'>Note: when using the source code, you should refer to my notes on page 33 of this topic</span>.
For those who aren't aware, this is an update of Merl's 1.7 Custom Build of Zoner's tools. All of the features that Merl and Zoner added are still available in this distribution.
<span style='color:red'>See the red notes below for important information on the latest build.</span>
<!--QuoteBegin-^Requiem^+Jan 26 2003, 08:26 PM--></div><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (^Requiem^ @ Jan 26 2003, 08:26 PM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin-->I see that a fix for a larger amount of maxplanes might be heading our way in the future, but I'm trying to discern if there is anything available to me right NOW to allow me to build a more compilcated level.<!--QuoteEnd--></td></tr></table><div class='postcolor'><!--QuoteEEnd-->
When you posted there wasn't, but I've gotten Merl's permission to post modified copies of version 1.7 with MAX_MAP_PLANES set to 65535. Replace your build tools with these versions and you'll be able to compile bsps with over 32767 planes. Then use my optimizer (included in the zip files) to make them safe for the game again. I'm putting them into two zip files since they're over the 122880 byte max file limit in a single download.
To use my optimizer, you can drag and drop your bsp file over it. The program overwrites the original file, so make a backup if you want to preserve the unoptimized copy. **Note -- since this program shrinks the map, it will checksum differently -- if a server is running the unoptimized copy and you have the optimized, it might not let you connect **.
Once the map is optimized, HLCSG -onlyents and HLRAD will break (make the optimization the final step when you're compiling), but it will run fine in the game.
<span style='color:gray'>EDIT: If you get a 'file not found' error when running the tools, look at Ollj's post below, he has a solution to the problem. You can get the DLL he talks about <a href='http://www.cyscape.com/download/msvcr70.dll' target='_blank'>here</a>. If you still can't get the tools to run, two people have reported needing a second DLL you can grab from <a href='http://www.dll-files.com/dllindex/dll-files.shtml?msvcp70' target='_blank'>here</a>, but others seem to be OK without it.</span>
<span style='color:gray'>EDIT 2/4/03: updated opt_plns to version 1.2, fixing the crash on bsp file not found and fixing a rare case of corrupted data for the first plane on the map.</span>
<span style='color:gray'>EDIT 2/5/03: updated opt_plns (in first download) again to version 1.3, fixing a crash bug above 32K planes and adding an option (-nopause) to skip the keypress prompt at the end of execution. The updated opt_plns is available as a separate program on the 4th page of this thread. Unless another bug is found, this should be the last update for a while.</span>
<span style='color:gray'>EDIT 2/8/03: well that was dumb of me <!--emo&???--><img src='http://www.unknownworlds.com/forums/html//emoticons/confused.gif' border='0' style='vertical-align:middle' alt='confused.gif' /><!--endemo-->. I posted the opt_plns zip instead of the modified tools 1 zip that includes the new opt_plns here -- thanks to [builder]hicks for helping me catch the problem.</span>
<span style='color:gray'>EDIT 2/11/03: updated opt_plns again to version 1.4, removing internal limits on texdata, visdata, and lightdata. Also added a -logfile <filename> switch and made input filename parsing more forgiving.</span>
<span style='color:gray'>EDIT 2/21/03: updated the tools to release 1.7p3, which speeds up HLCSG and fixes several clipping bugs as detailed <a href='http://xp-cagey.com/articles/clip_hull' target='_blank'>here</a>. There is a thread talking about the HLCSG changes <a href='http://www.unknownworlds.com/forums/index.php?act=ST&f=4&t=24137&st=0&' target='_blank'>here</a>. You'll want to download the second attachment again if you have grabbed these files before.</span>
<span style='color:gray'>EDIT 3/13/03: updated the tools to release 1.7p4, which contains a threadlock bugfix, code that is more friendly to concave brushes, and a new -cliptype normalized option. More details are available <a href='http://www.unknownworlds.com/forums/index.php?act=ST&f=4&t=24137&st=0&' target='_blank'>here</a>.</span>
<span style='color:gray'>EDIT 4/09/03: updated the tools to release 1.7p5, which added some new features and made some optimizations. More details are available <a href='http://www.unknownworlds.com/forums/index.php?act=ST&f=4&t=21248&st=105' target='_blank'>here</a>.</span>
<span style='color:gray'>EDIT 4/10/03: the 1.7p5 zipfiles contained the wrong version of the tools yesterday--my apologies to the 11 people who downloaded those files -- the links are now correct.</span>
<span style='color:gray'>EDIT 4/10/03: quick turnaround to version 1.7p6 to fix a show stopping bug related to the new BEVEL texture code in HLCSG.</span>
<span style='color:gray'>EDIT 4/13/03: released 1.7p7 added texlight support back into the tools and fixed problem of "too many direct lightstyle" message spam (you now will see the message once instead of every time the problem occurs). This is a stable release.</span>
<span style='color:gray'>EDIT 4/17/03: some people have had problems with the new lighting code, so I've added a switch, -oldmath, to RAD that allows using the older basic math functions. The new release has been tagged 1.7p8.</span>
<span style='color:gray'>EDIT 5/4/03: the -oldmath tag has been deprecated since I found the source of the problem. Also added support for the SDK 2.2 hull file format. Released as 1.7p9.</span>
<span style='color:gray'>EDIT 5/8/03: fixed a bug in opaque entity handling, released as 1.7p10.</span>
<span style='color:gray'>EDIT 8/15/03: maintenance release -- throw fatal error on buffer overflows that used to be silent, added location information to "too many lightsamples" error, using -verbose on HLRAD now displays ALL of those errors (there can be thousands in a single map). Added "-lightdata #" command so that people can add extra lightdata to their maps--doing this, like adding texdata, will require you to test your map on low-end machines to make sure that people can play your map. Released as 1.7p11</span>
<span style='color:gray'>EDIT 8/22/03: Rolling back publicly available release to p10 due to lighting bug report. Will release p11 again once the bug has been resolved.</span>
<span style='color:gray'>EDIT 10/3/03: Resolved the issue in p11, commented out spamming would-be developer message from HLRAD, reset default lightdata limit back to 6MB. Thanks to Kage and WolfWings for testing the release cantidates.</span>
<span style='color:gray'>EDIT 2/3/04: p13: Fixed bug in creation of objects without clip hulls, fixed bug in info_compile_parameters entity handler, changed usage of zhlt_noclip and zhlt_invisible to allow inclusion of those properties in .fgd files. Added Hullu's latest lighting enhancements.</span>
<span style='color:gray'>EDIT 6/1/04: p14: Fixed bug that caused errors building under default steam installs. Added switch to turn off light bounces for dynamic lights. Built optimizer into HLBSP. Raised internal plane limit to 256K. Added Reve's overhauled documentation.</span>
<span style='color:gray'>EDIT 6/3/04: p15: Fixed filehandler crash bug. Fixed face index crash bug. Added switch to turn off plane optimization. Fixed SolidBSP display times. Fixed bug with opaque entity lighting. Changed chart output to display planes percentage based on Half-Life plane limit.</span>
<span style='color:red'>EDIT 8/15/04: source release: I've released my latest source a few days ago since I've been hired by a game company and won't be working on ZHLT any more. Notes about the release are in the original posting on page 33 of this topic; I forgot to post a first-page update when I uploaded the source, but it's here now.</span>
Enjoy! <!--emo&:)--><img src='http://www.unknownworlds.com/forums/html//emoticons/smile.gif' border='0' style='vertical-align:middle' alt='smile.gif' /><!--endemo-->
<b>Latest download: </b> <a href='http://xp-cagey.com/downloads/zhlt_1.7p15.zip' target='_blank'>ZHLT 1.7p15</a>
Comments
Keep it up, this is one of the best additions to zoners! lets hope it gets official with the next release of ZHLT. You <!--emo&::onos::--><img src='http://www.natural-selection.org/forums/html/emoticons/tiny.gif' border='0' valign='absmiddle' alt='tiny.gif'><!--endemo--> .
my OS has missing the file
MSVCR70.DLL
too bad that the hammer-compiler does not tell me that, it just says "file not found".
Just another batch compiler told be about the missing dll, so I asked google about it...
<!--QuoteBegin--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> </td></tr><tr><td id='QUOTE'><!--QuoteEBegin-->
This DLL is needed by applications compiled to use the Microsoft
Foundation Classes (MFC) as a DLL, and compiled on a machine that has
the Microsoft .NET SDK installed on it. This is no different than older
MFC runtime DLL requirements; it's just that this is a very new runtime
DLL and most folks don't have it. The system running this application
(mapserver, compiled this way) also must have Microsoft Internet
Explorer 4.0 or higher installed on it, since the runtime uses
components from IE as well.
<!--QuoteEnd--></td></tr></table><span class='postcolor'><!--QuoteEEnd-->
You can get this file anywere on the internet.
Copy it in your main system directory or in system32 .
well,
<b>the compiling succeded:</b>
<!--QuoteBegin--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> </td></tr><tr><td id='QUOTE'><!--QuoteEBegin-->
Object names Objects/Maxobjs Memory / Maxmem Fullness
------------ --------------- --------------- --------
models 196/400 12544/25600 (49.0%)
<b>planes 31086/65535 621720/1310700 (47.4%)</b>
vertexes 4765/65535 57180/786420 ( 7.3%)
nodes 2504/32767 60096/786408 ( 7.6%)
texinfos 6504/32767 260160/1310680 (19.8%)
faces 3681/65535 73620/1310700 ( 5.6%)
clipnodes 12193/32767 97544/262136 (37.2%)
leaves 1771/8192 49588/229376 (21.6%)
marksurfaces 3926/65535 7852/131070 ( 6.0%)
surfedges 16234/512000 64936/2048000 ( 3.2%)
edges 8181/256000 32724/1024000 ( 3.2%)
texdata [variable] 2380/4194304 ( 0.1%)
lightdata [variable] 0/4194304 ( 0.0%)
visdata [variable] 0/2097152 ( 0.0%)
entdata [variable] 17394/524288 ( 3.3%)
54 textures referenced
=== Total BSP file data space used: 1357738 bytes ===
119.95 seconds elapsed [1m 59s]
<!--QuoteEnd--></td></tr></table><span class='postcolor'><!--QuoteEEnd-->
the map is half-completed (raw hallways) and already touches the 90% limit, because its BIG.
now its 50% and I hope the compression tool cuts enough planes from the void.
This is absolutely superb. I love you. <!--emo&:D--><img src='http://www.unknownworlds.com/forums/html/emoticons/biggrin.gif' border='0' valign='absmiddle' alt='biggrin.gif'><!--endemo-->
Now we may find out.
<!--emo&:0--><img src='http://www.unknownworlds.com/forums/html/emoticons/wow.gif' border='0' valign='absmiddle' alt='wow.gif'><!--endemo-->
So far it seems to work. A test map shrunk from 2080 planes to 484, and seems to run fine in-game.
Merl had bumped up the defined amount before, then knocked it back down when it caused in-game problems. This is a source comment embedded at the point of the directive:
<!--c1--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>CODE</b> </td></tr><tr><td id='CODE'><!--ec1-->#define MAX_MAP_PLANES 32767
// (from email): I have been building a rather complicated map, and using your latest
// tools (1.61) it seemed to compile fine. However, in game, the engine was dropping
// a lot of faces from almost every FUNC_WALL, and also caused a strange texture
// phenomenon in software mode (see attached screen shot). When I compiled with v1.41,
// I noticed that it hit the MAX_MAP_PLANES limit of 32k. After deleting some brushes
// I was able to bring the map under the limit, and all of the previous errors went away.<!--c2--></td></tr></table><span class='postcolor'><!--ec2-->
The problem has always been game engine based rather than something with the compiler. I suspect that the half-life engine uses a 32K sized static array for storing the loaded planes, and loses information when this is exceeded. There isn't anything magic about the 32K plane limit (it's not a hardware concern like the 4MB texture limit, for instance), and I'm willing to bet it was a semi-arbitrary size placed by the programmers when 400 was considered a high r_speeds value (akin to "nobody will ever need over 640K of memory" <!--emo&:)--><img src='http://www.unknownworlds.com/forums/html/emoticons/smile.gif' border='0' valign='absmiddle' alt='smile.gif'><!--endemo--> ).
i can even detail my map further now... god i wub j00
In my experience, if the map has too many planes in it then you can't fully compile the map anyway. If you've removed this 'lock', then kudos to you and congratulations on producing such a useful tool. But, if you still can't compile maps which originally have too many planes (before the optimisation), then that makes this kinda useless.
Yep, it's been moved up to about double what it was before (65535 now). If that's still not high enough, another recompile can shift it upwards again... the only limitation for the compiler tools (I'm <i>not</i> talking about running the game itself here) is the amount of memory required to store the information while it's being processed.
EDIT: shifting the limit upward again will require storing the information outside of the bsp during the compile because the bsp face structure's plane number field can only hold 16 bits.
Damn straight! <!--emo&:p--><img src='http://www.unknownworlds.com/forums/html/emoticons/tounge.gif' border='0' style='vertical-align:middle' alt='tounge.gif'><!--endemo-->
Theres an URL to the .DLL if you can't find it or want it instantly.
Theres an URL to the .DLL if you can't find it or want it instantly. <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
Thanks for tracking that down Rev -- added a copy of the link to the first post so people will see it <!--emo&:)--><img src='http://www.unknownworlds.com/forums/html/emoticons/smile.gif' border='0' style='vertical-align:middle' alt='smile.gif'><!--endemo-->
<a href='http://forum.hlrally.net/index.php?act=ST&f=3&t=2117' target='_blank'>http://forum.hlrally.net/index.php?act=ST&f=3&t=2117</a> - the url for u to look at, tell me if u want anything added as you are the official "compiler tool owner" seen as u made them
The only thing I would like added to that thread is a mention of the optimizer -- if you forget to use it and you're over the old planes limit, the map still breaks in the game... running the optimizer is really what allows the new flexibility-- ZHLT has had the higher limit before, and I don't want to cause any confusion. The extra program (opt_plns.exe) isn't part of the official tools, but I've submitted the source to Merl for inclusion in future builds.
NS_Bast 63.803212606119831429792951810908
32746 Before
11853 After
NS_Tanith 62.659141619635122719516996981231
30476 Before
11380 After
NS_Caged 59.534332219467500269483669289641
18554 Before
7508 After
NS_Eclipse 65.626992561105207226354941551541
18820 Before
6469 After
NS_Europa 62.807934307347765809960541751093
18754 Before
6975 After
NS_Hera 66.903478686918177364037236648702
32656 Before
10808 After
NS_Nancy 63.984801534602331415080419064483
27108 Before
9763 After
NS_Nothing 63.547775872958585220608138590634
22166 Before
8080 After
Average amount reduced: ~63%
Minimum amount reduced: ~59%
Maximum amount reduced: ~66%
Based on this, I'm seeing just shy of a 2/3rds reduction in the larger maps. Note that Hera and Eclipse, two of the maps that ran into the most problems in development as I recall, and likely had the most plane-optimization done to them already, benefited the most from the optimizer, while Caged didn't break 60% benefit, unlike all other maps in the test-set.
So, to be able to truly 'unlock' the abilities of higher plane-counts, I'd recommend supporting enough planes that in real-world work, even with the optimizer, it's possible to end up with too many planes. This may sound counter-productive, but right now you're possibly artificially-limiting the planes available now, with how effecient the optimizer actually is.
Also, this is a good example of just how much of a reduction this one tool is actually able to make in-game.