OpenWatcom: Fixing a Compiler Bug
Here I go through the story of finding a bug in a compiler, building, and fixing it. It’s a story of dealing with an unfamiliar build system and unfamiliar code base. Not all the details of the false starts and mistakes follow, but enough of them to show how one can use basic tools to figure out how an unfamiliar system works.
Why OpenWatcom? I first came across OpenWatcom when I started being interested in programming for OS/2. This was in 2006 or so. At the time, and still I believe, OpenWatcom was the only current compiler that made native OS/2 binaries. GCC compiled code requires a Posix runtime environment, either the old EMX, or the newer klibc1.
I’ve also used OpenWatcom for DOS development. I am interested in retro programming, and at the time (in the 90s) I just wasn’t good enough to do what I wanted to do; so I’ve let my teenage dreams come true in recent years. Mostly I run my DOS applications in DOSBox, and I run my compiler in the same system I use to run DOSBox, such as ArcaOS and Windows. OpenWatcom is famous for being the compiler of choice for most of the big name DOS games such as Doom and Tomb Raider; see the list of software made with the Watcom compilers.
Even though OpenWatcom only makes 32bit binaries, I’ve also used this occasionally for Windows programming. I don’t remember where and when, but I came across Structured Exception Handling in the context of Windows programming, and one day I wanted to give it a try; this time I chose OpenWatcom rather than Visual Studio.
Table of Contents
- The Erroneous Program
- Bulding OpenWatcom 1.9
- Making a debug build of the C compiler
- Debugging the C compiler
- Fixing OpenWatcom
- What Now?
- Source Patch
- Final Words
- Credits
- Social Media
- Installation
- Downloads
- Footnotes
The Erroneous Program
This all started very innocently. I found references to Structured Exception Handling in the OpenWatcom manuals, and decided to try it. At the time I was using OpenWatcom from the version 2 GitHub.
For people interested in it, this is a Microsoft extension to C and C++, built into the operating system, and the documents can be found online.
For more information about how this is handled at the operating system level, and how to mix C++ and SEH exceptions, see this article on CodeProject.
As can be seen in the screenshot above, only the OpenWatcom C compiler, and not the C++ compiler supports Structured Exception Handling, unlike Microsoft’s C++ compiler2.
So let’s start with my example program.
#include <stdio.h>
int main( int argc, char *argv[] )
{
__try {
printf( "Hello, world.\n" );
}
return 0;
}
Let’s call this file seh_err.c.
An experienced Windows programmer will immediately know what’s wrong with the above code, but please bear with me, I didn’t know at the time.
My sources files are in the c:\src\c\ow\nt directory, short for “sources”, “c”, “openwatcom”, “windows nt”. For those who don’t know Windows 10 comes from a family of operating systems named Windows NT, originally released in 1993. You can read more about the history on WikiPedia.
This story takes place is several cmd windows3. Because I use many compilers, I chose not to make a “system wide” installation of OpenWatcom, instead, I purposefully open a cmd window, and run the environment setup batch file; I never use the IDE4.
To test the above source file, I opened a new cmd window for OpenWatcom 2, and ran that batch file; let’s call this window cmd0. I cded to my source file,
cd c:\src\c\ow\nt
and, because my OpenWatcom 2 compiler is in c:\watcom, I did
c:\src\c\ow\nt> \WATCOM\owsetenv.bat
which gives the following output,
Open Watcom Build Environment
and then tried to compile my file.
c:\src\c\ow\nt> wcl386 seh_err.c
which got me the following message.
Open Watcom C/C++ x86 32-bit Compile and Link Utility
Version 2.0 beta Sep 29 2019 01:01:24 (64-bit)
Copyright (c) 2002-2019 The Open Watcom Contributors. All Rights Reserved.
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
wcc386 seh_err.c
Open Watcom C x86 32-bit Optimizing Compiler
Version 2.0 beta Sep 29 2019 00:45:35 (64-bit)
Copyright (c) 2002-2019 The Open Watcom Contributors. All Rights Reserved.
Portions Copyright (c) 1984-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
seh_err.c: 10 lines, included 799, 0 warnings, 0 errors
^C
c:\src\c\ow\nt>
So the compiler hangs, and it appears everything is working normally. The ^C above is when I pressed ctrl+c to stop the compiler after giving up waiting. Further experiments which I will not detail here showed that OpenWatcom 2 could create a (corrupt?) object file, but wouldn’t create an executable and there was no information about the reason.
Not willing to give up, I decided to download OpenWatcom 1.9 from the openwatcom.org website. Once installed, I gave it another try. In another cmd window, which we shall call cmd1, I did,
cd c:\src=c\ow\nt
C:\src\c\ow\nt> \opt\WATCOM19\owsetenv.bat
Open Watcom Build Environment
because my OpenWatcom 1.9 compiler is in c:\opt\watcom19. Now I tried to compile my source file again,
C:\src\c\ow\nt> wcl386 seh_err.c
which gave me the following error message.
Open Watcom C/C++32 Compile and Link Utility Version 1.9
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
wcc386 seh_err.c
Open Watcom C32 Optimizing Compiler Version 1.9
Portions Copyright (c) 1984-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
seh_err.c: 10 lines, included 771, 0 warnings, 0 errors
The instruction at 0x694351f7 referenced memory at 0x0000001c.
The memory could not be read.
Exception fielded by 0x00403560
EAX=0x00000000 EBX=0x00000037 ECX=0x00000000 EDX=0x00000000
ESI=0x00000000 EDI=0x694b1f38 EBP=0x00000000 ESP=0x0019f8c4
EIP=0x694351f7 EFL=0x00010246 CS =0x00000023 SS =0x0000002b
DS =0x0000002b ES =0x0000002b FS =0x00000053 GS =0x0000002b
Stack dump (SS:ESP)
0x00000001 0x004153c0 0x00000000 0x00000000 0x694b1f38 0x0202a610
0x00000003 0x0202a5f0 0x00000001 0x69434388 0x0019fe48 0x00000000
0x0019fe1c 0x0000001c 0x0000001c 0x004142d0 0x694345e4 0x69434713
0x0019fbfc 0x0019fc04 0x0019fbd8 0x0019fe1c 0x00000000 0x0019fe48
0x69434616 0x0019f90c 0x002b002b 0x00530023 0x002b002b 0x0019ff5c
0x694107a4 0x0019f954 0x0019fbd8 0x0019fc04 0x0019fbfc 0x69403124
0x0019fbfc 0x0019fc04 0x0019fbd8 0x0019fe1c 0x00000000 0x0019fe48
0x69402f82 0x0019f954 0x002b002b 0x00530023 0x002b002b 0x0019ff5c
0x0019fbfc 0x0019fbd8 0x69402b75 0x00000000 0x694483c0 0x0019f9c8
0x7789172b 0x00000001 0x69400000 0x00000000 0x00000000 0x0019f9c0
0x005f6260 0x69400000 0x00401095 0x69400090 0x01400000 0x0019f9e0
0x7788bf5b 0x694b9000 0x69400000 0x00401095 0x00000006 0x0019fa10
Error: Unable to invoke "C:\OPT\WATCOM19\BINNT\wcc386.exe"
C:\src\c\ow\nt>
This looks much more promising. We have an access violation, or a segmentation fault (segfault) and that tells us there really is a bug in the compiler.
Now at this time, I decided to see if I could build a debug version of the compiler. The first thing I did, was to find and look at the Contributor Resources.
The OpenWatcom site has a link conveniently named Wiki, which I had browsed several years ago, and I remembered it had lots of developer related resources.
On the main wiki page, we have a link to the contributor resources on the left side.
On the contributor resources page, the three most important links are build architecture, compiler architecture, and compiler debugging.
As you can see, these are the only three pages I’ve looked at in my history. Most of build architecture is irrelevant to building OpenWatcom for the first time, but at the very bottom there’s mention of an overall build tool, called builder, and while I did not use any of the things discussed in this page, the mention of the builder tool helped me make sense of the top level scripts later on.
The Compiler Architecture page is really helpful. It points to the directory within the source that holds the C front end, bld/cc, and the code generator, bld/cg. At this time I had no idea if the bug I had stumbled upon was in the code generator or front end, so I kept my mind open. It also mentions some notable files in both the frond end and back end; notably cstmt.c and cgen.c in the front end, which turned out to be important later.
The Compiler Debugging article is my main source of information. It points to the debug build directories of both the C front end and the code generator back end; pointing to bld/cc/dnt386.386 and bld/cg/intel/386/nt386.dbg respectively. There are details about getting information from the frond end and code generator in a debug environment, but none of that turned out to be used in the following story.
After reading that, I was fairly confident I could make a debug build if I got the compiler to build in the first place.
Building OpenWatcom 1.9
This part of the story takes place in yet another cmd window, because I wanted to keep my history in the other two; let’s call it cmd2. Again, I started by running the OpenWatcom 1.9 environment script.
C:\> \opt\WATCOM19\owsetenv.bat Open Watcom Build Environment
After downloading the 1.9 source code, I extracted it into my c:\src directory.
cd c:\src 7z x %homepath%\Downloads\open_watcom_1.9.0-src.zip
Now I had an OW19 directory with the contents shown by dir/w.
Directory of c:\src\OW19
[.] [..] [bat] [bld] bootclean.sh
build.cmd build.sh buildrex.cmd clean.sh cmnvars.bat
cmnvars.cmd cmnvars.sh [contrib] [distrib] [docs]
license.txt myvars.cmd owconfig.bat owconfig.vbs projects.txt
readme.txt [rel2] setvars.bat setvars.cmd setvars.dos
setvars.sh zipup-rel.cmd zipup-rel.sh zipup.bat zipup.sh
22 File(s) 65,969 bytes
The above listing has one file generated by me, myvars.cmd, and we’ll come to that later.
The readme.txt file includes information about the build system, but has no words about bootstrapping the build process. For that we have to resort to the scripts we can see mentioned above.
We can see some promising scripts there, called build.cmd, and build.sh. The build.cmd starts with this helpful comment.
REM ***************************************************************** REM build.cmd - build Open Watcom using selected compiler REM REM will build the builder, wattcp, watcom and installer REM REM combined OS/2, eCS and Windows version REM REM If first argument is "self", uses tools in rel2 to build, REM requiring customized devvars.cmd. Otherwise, customized REM myvars.cmd is needed. If the appropriate file does not exist, REM owconfig.bat will be invoked to automatically generate it. REM If running on OS/2 or eCS, it has to be created manually. REM REM Call without parms for "builder rel2" operation -> build REM Call with clean for "builder clean" operation -> build clean REM --> requires a customized setvars.bat/cmd named myvars.cmd REM --> set WATCOM to the existing OW 1.x installation REM *****************************************************************
So, what is this file myvars.cmd all about? Unfortunately, the comment doesn’t say, and neither does the Build Architecture wiki page.
What it turns out to mean, you’re supposed to choose one of the setvars scripts, rename it to myvars.cmd and then run the build script. What I ended up doing, because the comment above specifically mentions myvars.cmd, was to rename the OS/2 script and then change it to Windows. Presumably, it would have been easier to just rename setvars.bat to myvars.cmd, but I didn’t do that.
So, to recap, I’ll go through my original steps, rather than explain “how to do it properly.” That should preferably be added to the OpenWatcom wiki somewhere.
copy setvars.cmd myvars.cmd
Then edit the myvars.cmd file and change the following settings.
REM Change this to point to your Open Watcom source tree set OWROOT=c:\src\ow19 REM Change this to point to your existing Open Watcom installation set WATCOM=c:\opt\watcom19
And at the bottom of the file, refer to the Windows common settings, rather than the OS/2 ones. Notice that the following changes .cmd to .bat.
REM Invoke the script for the common environment call %OWROOT%\cmnvars.bat
I figured most of that out by reading the source code of these various scripts, and a few false starts.
Now, we can build OpenWatcom with a single command.
build
On 64bit Windows 10, the build will halt with the following message.
Unsupported 16-Bit Application The program or feature "\??\c\src\ow19\docs\gml\dos\wgml.exe" cannot start or run due to incompatibility with 64-bit versions of Windows. Please contact the software vendor to ask if a 64-bit Windows compatible version is available.
A side note. The OpenWatcom developers are recreating wgml so the next release should run on 64bit platforms.
This just means the documentations aren’t created and for this exercise we don’t care.
Press [OK] to continue the build without documentation.
You will also see this in the build log, which just reiterates the message above.
This version of c:\src\ow19\docs\gml\dos\wgml.exe is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher.
Making a debug build of the C compiler
Now we can turn to the task of building a debug version of the C compiler. This is where the Compiler Debugging wiki page is indispensable. Particularly the section titled Debug Builds of Compilers, where the above mentioned debug build directories are.
First, we take a look at the bld\cc\dnt386.386 directory,
cd bld\cc\dnt386.386 c:\src\OW19\bld\cc\dnt386.386> type makefile
where we have the following makefile.
#pmake: debug 386 nt cpu_386 os_nt target_386 host_os = nt host_cpu = 386 cc_release = 0 debug_cg = 1 !include ../targ_386.mif
This looks promising. We see that it’s configured to pick up the debug build of the code generator, with debug_cg = 1 so we’ll start there,
cd ..\..\cg\intel\386\nt386.dbg c:\src\OW19\bld\cg\intel\386\nt386.dbg> type makefile
where we have the following makefile.
#pmake: debug os_nt cpu_386 386 intel target_386 host_os = nt host_cpu = 386 cg_release = 0 !include ../mif/master.mif
And this looks promising too. So we try to build it.
wmake
And that gives us the following error message.
Open Watcom Make Version 1.9
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
bwcl386 -l=nt -IC:\OPT\WATCOM19\h -Ic:\src\ow19\bld\watcom\h -zq
c:\src\ow19\bld\watcom\c\bitset.c -fpi -ox -w4 -fe=.\bitset.exe
Error(E14): Cannot execute (bwcl386): No such file or directory
Error(E42): Last command making (.\bitset.exe) returned a bad status
Error(E02): Make execution terminated
Now, what is this bwcl386 command? It’s not part of the OpenWatcom 1.9 release, so we have to dig further. For this, we bring up the big guns, Unix find. It so happens that I have the Mozilla build tools installed, where I have lots of Unix tools, so I open another command shell, cmd, and start it; let’s call this shell cmd3.
\mozilla-build\start-shell.bat
And go to the source directory, using Unix shell syntax.
cd /c/src/OW19/
And apply find to the problem.
find . -name 'bwcl386*'
Presumably I could also use dir/s for this, but I’m used to find by now. This resulted in
./bld/build/binnt/bwcl386.exe
and that’s a clue that we can add this directory to our path and try to build the code generator again. Returning to the previous command line, in cmd2, we add it, and try again.
path c:\src\OW19\bld\build\binnt;%path% wmake
And now it builds.
We go back to the C compiler, and build try to build that.
cd ..\..\..\..\cc\dnt386.386 wmake
And that builds too. We see that the last command in the build output is
wlink op q name wcc386c.exe @exe.lnk
so we know that the final executable is called wcc386c.exe. We can verify this with
dir/w *.exe
which shows
Directory of c:\src\OW19\bld\cc\dnt386.386
findhash.exe intlbin.exe mkcode.exe mkopcod1.exe mkopcode.exe msgencod.exe
usageenc.exe wcc386c.exe
8 File(s) 1,338,880 bytes
So now we have a debug build of the C compiler. Let’s try it in the cmd1 window with our source code,
C:\src\c\ow\nt> \src\OW19\bld\cc\dnt386.386\wcc386c.exe seh_err.c
which spews the same error messsage again.
Open Watcom C32 Optimizing Compiler Version 1.9 Portions Copyright (c) 1984-2002 Sybase, Inc. All Rights Reserved. Source code is available under the Sybase Open Watcom Public License. See http://www.openwatcom.org/ for details. seh_err.c: 10 lines, included 771, 0 warnings, 0 errors The instruction at 0x00443a0b referenced memory at 0x0000001c. The memory could not be read. Exception fielded by 0x004b7a20 EAX=0x00000000 EBX=0x00000001 ECX=0x000000a6 EDX=0x00000000 ESI=0x021fa610 EDI=0x004fbf30 EBP=0x0019fcc8 ESP=0x0019fca0 EIP=0x00443a0b EFL=0x00010246 CS =0x00000023 SS =0x0000002b DS =0x0000002b ES =0x0000002b FS =0x00000053 GS =0x0000002b Stack dump (SS:ESP) 0x00000000 0x021fffec 0x005153c0 0x00000001 0x004fbf37 0x021faa00 0x004424b5 0x00000000 0x00000000 0x00000000 0x0019fcfc 0x004fbf30 0x021fa610 0x00000003 0x000000a6 0x00000001 0x0044266c 0x021fa6b8 0x021ffff8 0x021ff7a4 0x021fa5f0 0x00000000 0x00000000 0x0019fd20 0x0047291a 0x0047291a 0x00514210 0x000000a6 0x00000001 0x00442a12 0x00000001 0x021fa5f0 0x0019fd7c 0x0047291a 0x0047291a 0x00514210 0x000000a6 0x00000001 0x00442b7e 0x00000108 0x0019fe54 0x00000000 0x0047291a 0x0047291a 0x0019fd7c 0x00442a68 0x0019fd3c 0x002b002b 0x00530023 0x002b002b 0x0019ff5c 0x00000108 0x0019fd98 0x00405432 0x00008409 0x0019fdd4 0x0047291a 0x0047291a 0x00000000 0x0019fe54 0x00000108 0x00403acf 0x00000108 0x0019fe54 0x00000000 0x0047291a 0x0047291a 0x0019fdd4 0x0040395a 0x0019fd98 0x002b002b 0x00530023
No surprise there.
Debugging the C compiler
We now apply the debugger to the C compiler. It isn’t documented in the manual, but we can apply the debugger to a command and its arguments like this5.
wdw \src\OW19\bld\cc\dnt386.386\wcc386c.exe seh_err.c
This debugs the compiler we just built, when applied to the argument seh_err.c.
And we see that the debugger starts up with the main() function showing.
int main( void )
{
char *argv[2];
int ret;
char *buffer;
int len;
#else
When we press the [go] button, the picture of a man running, we get this message.
After pressing [ok], you’re presented with the following source code.
This shows you a piece of cgen.c, which we know is in the front end, because of the wiki page we read earlier.
for( try_index = 0; try_index <= max_try_index; try_index++ ) {
stmt = ValueStack[ try_index ];
=> DGInteger( stmt->op.st.parent_scope, T_UINT_1 ); // parent index
if( stmt->op.opr == OPR_EXCEPT ) {
DGInteger( 0, T_UINT_1 );
} else {
DGInteger( 1, T_UINT_1 );
}
except_label = FEBack( stmt->op.st.try_sym_handle );
DGBackPtr( except_label, FESegID( CurFuncHandle ), 0, T_CODE_PTR );
}
We see that the compiler is stopped in a line where stmt is referenced, and it’s NULL in the local variable list.
Locals ====== old_segment 1 except_label NULL except_table 0x001D53C0 try_backinfo 0x0229FFEC stmt NULL <== try_index 0 max_try_index 0 tree 0x0229AA00
Now, the first thing we might want to do, is to add some sort of NULL pointer check, and propagate the error to the interface somehow. But first, let’s examine what’s really going on here.
Side rant. One thing I noticed with wdw and that is it has apparently no feature to open the source file in a text editor, or even tell you where it is. This is really annoying if you want to look at the file in an external tool.
We have a loop, using the variable try_index, which sets stmt to some value on the ValueStack. This is the line immediately above the faulty line.
So, what is on the ValueStack? Let’s examine the code and find out.
Above this, in the function, we find
for( ; tree != NULL; tree = tree->left ) {
stmt = tree->right;
if( stmt->op.opr == OPR_FUNCEND )
break;
switch( stmt->op.opr ) {
case OPR_TRY:
try_index = stmt->op.st.try_index;
if( try_index > max_try_index )
max_try_index = try_index;
break;
case OPR_EXCEPT:
case OPR_FINALLY:
ValueStack[ try_index ] = stmt;
break;
default:
break;
}
}
and we see that ValueStack[] is added to when the compiler sees the __except or __finally keywords. So what we’re dealing with here, might be a syntax error. Let’s check what Microsoft’s cl has to say about seh_err.c. We open up x64 Native Tools Command Prompt for VS 2019. Tip: start typing x64 into the search box in Windows 10, and after a while it’ll suggest the command prompt; if it’s your first time you have to wait for a noticeably long time. Let’s call this shell cmd4.
cd c:\src\c\ow\nt cl seh_err.c
And it gives us a syntax error message.
Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64 Copyright (C) Microsoft Corporation. All rights reserved. seh_err.c seh_err.c(9): error C2059: syntax error: 'return'
So this is indeed a syntax error, but the message from cl is not helpful. Let’s take a look at the source code again.
#include <stdio.h>
int main( int argc, char *argv[] )
{
__try {
printf( "Hello, world.\n" );
}
return 0;
}
And then fix it by adding a __finally clause.
#include <stdio.h>
int main( int argc, char *argv[] )
{
__try {
printf( "Hello, world.\n" );
}
__finally { }
return 0;
}
Let’s call this file seh_fix.c and try again.
cl seh_fix.c
This gives us a successful compilation.
Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64 Copyright (C) Microsoft Corporation. All rights reserved. seh_fix.c Microsoft (R) Incremental Linker Version 14.25.28612.0 Copyright (C) Microsoft Corporation. All rights reserved. /out:seh_fix.exe seh_fix.obj
And we try again with the OpenWatcom debug build we did earlier; in our cmd1 window.
\src\OW19\bld\cc\dnt386.386\wcc386c.exe seh_fix.c
This time we did not run it in the debugger. And we get a successful compile.
Open Watcom C32 Optimizing Compiler Version 1.9 Portions Copyright (c) 1984-2002 Sybase, Inc. All Rights Reserved. Source code is available under the Sybase Open Watcom Public License. See http://www.openwatcom.org/ for details. seh_fix.c: 12 lines, included 771, 0 warnings, 0 errors Code size: 76
Fixing OpenWatcom
Now that we know the compiler breaks on a syntax error, how do we fix it? What we want to know is where in the source files __try is handled. For this we use Unix find. In the C compiler directory, let’s look for __try; using our Mozilla build environment shell; this is our cmd3 window.
cd /c/src/OW19/bld/cc
find . -name '*.c' -exec grep -i __try '{}' +
What does this mean? Let’s look at each argument one by one;
findthis is the command we execute,.this is the directory we start the search in,-namethis tellsfindwe want to look only for certain file names,'*.c'we have to escape this from the shell, and it looks for C source files,-execthis tellsfindto execute a command for every filename found,grepthis is the command we run,-ithis tells grep to search case insensitively,__trythis is the regular expression we want to find,'{}'this tellsfindwhere to put the filename in argument list ofgrep, and we need to escape this from the shell again,+this tellsfindwhere to end the command line forgrep; we could have used a semi-comma';'also, which we’d have to escape from the shell, but that meansgrepwould be run with only one file
name per invocation, a+here means “more than one file is ok.”
What we get back is
./c/cdata.c: SymTryInit = 0; /* builtin symbol for '__TryInit' */
./c/cdata.c: SymTryFini = 0; /* builtin symbol for '__TryFini' */
./c/cdata.c: SymTryUnwind = 0; /* builtin symbol for '__TryUnwind' */
./c/cgen.c: CallTryFini(); // generate call to __TryFini
./c/cgen.c: CallTryInit(); // generate call to __TryInit
./c/cmodel.c: PreDefine_Macro( "_try=__try");
./c/cstmt.c: if( (block->block_type == T__TRY) || (block->block_type == T___TRY) )
./c/cstmt.c: if( (block->block_type == T__TRY) || (block->block_type == T___TRY) ) {
./c/cstmt.c: if( (block->block_type == T__TRY) || (block->block_type == T___TRY) )
./c/cstmt.c: if( ( block->block_type == T__TRY )
./c/cstmt.c: || ( block->block_type == T___TRY ) ) {
./c/cstmt.c: CurToken = T__TRY;
./c/cstmt.c: case T__TRY:
./c/cstmt.c: case T___TRY:
./c/cstmt.c: case T__TRY:
./c/cstmt.c: case T___TRY:
./c/csym.c: SymTryInit = MakeFunction( "__TryInit", typ ); /* 05-dec-92 */
./c/csym.c: SymTryFini = MakeFunction( "__TryFini", typ ); /* 05-dec-92 */
./c/csym.c: SymTryUnwind = MakeFunction( "__TryUnwind", typ ); /* 16-apr-94 */
and we see that the most promising file is cstmt.c. In retrospect, we could have guessed this, based on the wiki page we read earlier. So we load it up in an editor and look at it. I won’t go into the details of my code inspection of this file. Suffice to say, my editor has incremental search which I used for __try, and it’s fortunately case insensitive. I spent quite a few hours on the code inspection part and since I was not working on this project full time, I did my code inspection over two days6, and I eventually came to the conclusion there were two candidates for adding an error message to the compiler. The first is the function
static void EndOfStmt( void )
{
do {
switch( BlockStack->block_type ) {
//...
#ifdef __SEH__
case T__TRY:
case T___TRY:
if( EndTry() )
return;
break;
case T__EXCEPT:
case T___EXCEPT:
DropBreakLabel();
TryScope = BlockStack->parent_index;
CompFlags.exception_handler = 0;
break;
case T__FINALLY:
case T___FINALLY:
AddStmt( LeafNode( OPR_END_FINALLY ) );
CompFlags.in_finally_block = 0;
TryScope = BlockStack->parent_index;
break;
#endif
//...
and the second is the function
static int EndTry( void )
{
//...
if( (CurToken == T__EXCEPT) || (CurToken == T___EXCEPT) ) {
//...
return( 1 );
} else if( (CurToken == T__FINALLY) || (CurToken == T___FINALLY) ) {
//...
return( 1 );
}
return( 0 );
}
I’ve elided the code that’s not important for our discussion. Here we see that the code returns 1, or true, when the following token, aptly named CurToken for some reason, is __except or __finally. This is exactly the condition we want to use to add an error message to the C compiler. So we have a choice, of adding a new error to EndTry() or EndOfStmt(). EndTry() is called exactly once, so it doesn’t matter which one is chosen.
I have no particular stake in the matter. I’m not an OpenWatcom contributor (at the time of this writing) and I have no opinion on which call site would be “correct” in this code base. I will share this article with the contributors, and they can decide to accept my changes, or apply their own.
I chose EndOfStmt() because it looks like the if statement should have an else. Now, how do we test this? If we look for err in the code, we quickly find statements like
CErr1( ERR_INVALID_TYPE_FOR_SWITCH );
and we can oppropriate it to see if the modifications do what we want. So we now have
if( EndTry() )
return;
else
CErr1( ERR_INVALID_TYPE_FOR_SWITCH );
break;
and we build it again, in cmd2, with
c:\src\OW19\bld\cc\dnt386.386> wmake
but we get
Open Watcom Make Version 1.9
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
cc cstmt.obj
wlink op q name wcc386c.exe @exe.lnk
Error! E3008: cannot open wcc386c.exe : Permission denied
Error! E2010: I/O error processing wcc386c.exe : Permission denied
Error(E42): Last command making (wcc386c.exe) returned a bad status
Error(F43): Deleting (wcc386c.exe): Permission denied
Error(E02): Make execution terminated
This is because we’re on Windows, which when it’s running a program, locks the file. This is not a problem on modern Unix systems, or maybe has never been a problem7.
So, we just close the debugger which is still open, and try again,
c:\src\OW19\bld\cc\dnt386.386> wmake
and this time it works.
Open Watcom Make Version 1.9
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
wlink op q name wcc386c.exe @exe.lnk
Now we test again in our cmd1 window.
C:\src\c\ow\nt> \src\OW19\bld\cc\dnt386.386\wcc386c.exe seh_err.c
and get,
Open Watcom C32 Optimizing Compiler Version 1.9 Portions Copyright (c) 1984-2002 Sybase, Inc. All Rights Reserved. Source code is available under the Sybase Open Watcom Public License. See http://www.openwatcom.org/ for details. seh_err.c(9): Error! E1078: Invalid type for switch expression seh_err.c: 10 lines, included 771, 0 warnings, 1 errors
perfect! We got the error we specified, and the line number is for the return statement; which is as expected. Let’s look at our source code again, with line numbers this time.
1 #include <stdio.h>
2
3 int main( int argc, char *argv[] )
4 {
5 __try {
6 printf( "Hello, world.\n" );
7 }
8
9 return 0;
10 }
Now, how do we improve the error message? This is the 21st century, and we make helpful error messages, not just syntax error because we’re too lazy to implement context specific checks in the compiler. I’m looking at you, Visual Studio C.
The first thing we do is to figure out how to add our own error message. Let’s see if we can find ERR_INVALID_TYPE_FOR_SWITCH in a header file. We can use find for this again. In cmd3 the /c/src/OW19/bld/cc directory, we run
find . -name '*.h' -exec grep ERR_INVALID_TYPE_FOR_SWITCH '{}' +
and get nothing at all. That’s not good. Let’s try again and instead of limiting to header files, we use -type f for regular files.
find . -type f -exec grep ERR_INVALID_TYPE_FOR_SWITCH '{}' +
Some binary files match, but I’ve cut those out from the display.
./c/cstmt.c: CErr1( ERR_INVALID_TYPE_FOR_SWITCH ); ./c/cstmt.c: CErr1( ERR_INVALID_TYPE_FOR_SWITCH ); ./dnt386.386/msgdefs.gh:MSG_DEF( ERR_INVALID_TYPE_FOR_SWITCH , Errs , ERROR , 0 , 78 )\ ./dos386.386/msgdefs.gh:MSG_DEF( ERR_INVALID_TYPE_FOR_SWITCH , Errs , ERROR , 0 , 78 )\ ./dos386.i86/msgdefs.gh:MSG_DEF( ERR_INVALID_TYPE_FOR_SWITCH , Errs , ERROR , 0 , 78 )\ ./gml/cerrs.gml::MSGSYM. ERR_INVALID_TYPE_FOR_SWITCH ...
Here, each target has a matching msgdefs.gh file, which may be a generated header file. The interesting file is cerrs.gml, last line before I cut the output. Let’s take a look.
After the copyright notice, we have this comment,
:cmt * Description: C compiler diagnostic messages.
so this looks really promising. Following that we have some explanations about the syntax of the file. Following that, we have actual diagnostic definitions, starting with
:MSGGRP. Warn1 :MSGGRPSTR. W :MSGGRPNUM. 100 :MSGGRPTXT. Warning Level 1 Messages
so we use the editor search feature to find err. That isn’t very helpful, because most constants defining error messages start with ERR_. Similarly, error is equally unhelpful because a lot of documentation text contains that word. Looking at the code snippet above, we try msggrp, which quickly enough gives us line 706.
:MSGGRP. Errs :MSGGRPSTR. E :MSGGRPNUM. 1000 :MSGGRPTXT. Error Messages
So here we are looking at error message definitions. There is one concern when we add an error message. Ideally we shouldn’t change the numbers of errors already present in the code. Particularly since what we want here is a minor patch, not a major release. And even so, renumbering errors is a bad form. People search for, and reference error numbers all the time. It’s really impolite if we go around renumbering them.
So we try adding our new error message at the end of this section, searching for msggrp again to find the end. We find
:eMSGGRP. Errs
and immediately above that, we try to add our new error constant.
:MSGSYM ERR_TRY_NEEDS_EXCEPT_OR_FINALLY :MSGTXT A __try statement must be followed by either __except or __finally. .np This error message is undocumented.
As our first attempt8, we just keep the documentation texts simple, and say that the error message is undocumented. Not really helpful, but good enough for a test.
Now we try to build the compiler again, in cmd2,
c:\src\OW19\bld\cc\dnt386.386> wmake
And get,
Open Watcom Make Version 1.9
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
.\msgencod.exe -i -ip -q -g ../gml/cerrs.gml msgtxt.gh msgdefs.gh msgattr.gh
../gml/cerrs.gml(2171): Error! E000: tag missing '.': MSGSYM
../gml/cerrs.gml(2172): Error! E000: tag missing '.': MSGTXT
../gml/cerrs.gml(2350): Error! E000: fatal: cannot continue due to errors
Error(E42): Last command making (msgtxt.gh) returned a bad status
Error(E02): Make execution terminated
oops, I didn’t realize we had to terminate the keyword with a period. Let’s try to fix it.
:MSGSYM. ERR_TRY_NEEDS_EXCEPT_OR_FINALLY :MSGTXT. A __try statement must be followed by either __except or __finally. .np This error message is undocumented.
And run again.
c:\src\OW19\bld\cc\dnt386.386> wmake
Open Watcom Make Version 1.9
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
.\msgencod.exe -i -ip -q -g ../gml/cerrs.gml msgtxt.gh msgdefs.gh msgattr.gh
../gml/cerrs.gml(2171): Error! E000: MSGSYM ERR_TRY_NEEDS_EXCEPT_OR_FINALLY has no Japanese text
../gml/cerrs.gml(2350): Error! E000: fatal: cannot continue due to errors
Error(E42): Last command making (msgtxt.gh) returned a bad status
Error(E02): Make execution terminated
So we really need Japanese also; but this is what the comments at the top of the file had to say about Japanese.
:cmt Japanese error messages are supported via the :MSGJTXT tag. :cmt If there is no :MSGJTXT. for a particular :MSGSYM. then the :cmt message will come out in English.
So it appeared we wouldn’t have to add a Japanese error message, but in fact we do.
Now, I don’t know enough Japanese to translate the error message. Furthermore, I don’t have a shift-jis capable editor at hand9. And I certainly don’t want to set my locale to shift-jis for a single file. The Japanese messages show up as gibberish in my editor.
Now, here we take advantage of shift-jis. It’s basically backwards compatible with ASCII10. So we just copy the message into msgjtxt.
:MSGSYM. ERR_TRY_NEEDS_EXCEPT_OR_FINALLY :MSGTXT. A __try statement must be followed by either __except or __finally. :MSGJTXT. A __try statement must be followed by either __except or __finally. .np This error message is undocumented.
And we try again.
c:\src\OW19\bld\cc\dnt386.386> wmake
And that worked.
Open Watcom Make Version 1.9
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
.\msgencod.exe -i -ip -q -g ../gml/cerrs.gml msgtxt.gh msgdefs.gh msgattr.gh
.\intlbin.exe wcc386
cc ascii.obj
...
cc cintmain.obj
wlink op q name wcc386c.exe @exe.lnk
So now we have a working error message — hopefully. Let’s try to use it.
We return to our code in cstmt.c and try.
if( EndTry() )
return;
else
CErr1( ERR_TRY_NEEDS_EXCEPT_OR_FINALLY );
break;
Which we compile again with.
c:\src\OW19\bld\cc\dnt386.386> wmake
And get a successful compilation.
Open Watcom Make Version 1.9
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
cc cstmt.obj
wlink op q name wcc386c.exe @exe.lnk
So now we try it on our faulty source code, this time in cmd1,
C:\src\c\ow\nt> \src\OW19\bld\cc\dnt386.386\wcc386c.exe seh_err.c
and it works exactly as we want it to.
Open Watcom C32 Optimizing Compiler Version 1.9 Portions Copyright (c) 1984-2002 Sybase, Inc. All Rights Reserved. Source code is available under the Sybase Open Watcom Public License. See http://www.openwatcom.org/ for details. seh_err.c(9): Error! E1190: A __try statement must be followed by either __except or __finally. seh_err.c: 10 lines, included 771, 0 warnings, 1 errors
Now let’s try it on the fixed source code as well.
C:\src\c\ow\nt> \src\OW19\bld\cc\dnt386.386\wcc386c.exe seh_fix.c
And that works too.
Open Watcom C32 Optimizing Compiler Version 1.9 Portions Copyright (c) 1984-2002 Sybase, Inc. All Rights Reserved. Source code is available under the Sybase Open Watcom Public License. See http://www.openwatcom.org/ for details. seh_fix.c: 12 lines, included 771, 0 warnings, 0 errors Code size: 76
And just for completeness, we run the results.
C:\src\c\ow\nt> seh_fix.exe
And get.
Hello, world.
So far so good. It appears the change did not introduce any new bugs.
What Now?
There are two tasks left. One is to build a production version of the C compiler we can use to then patch OpenWatcom 1.9 installations. Alternatively, we could build new installers, but I don’t have a workaround yet for the 16bit wgml tool11, so the documentation will be missing.
The second task is to write proper documentation for the new error message.
Let’s start with the first task, and build new C compilers. Yes, there are plurals since we have binaries for DOS, Linux, OS/2, and Windows. In cmd2 we do,
c:\src\OW19> build
which will take a while, and stop with the same 16bit error message mentioned above, so just press [ok] and it’ll continue.
Now we have a new C compiler and the related dynamic link libraries in c:\src\OW19\rel2. This build includes compilers for PowerPC, MIPS, and I believe Alpha, which we have no reason to include in a patch.
In order to solve this, we turn to the Unix tool find again. This time, we make sure we’ve set up the OpenWatcom build environment first. This means, for me, that I have to do in a new cmd window, these commands; let’s call it cmd5.
\opt\WATCOM19\owsetenv.bat \mozilla-build\start-shell.bat
In bash I can now do
echo $WATCOM
and see
C:\OPT\WATCOM19
which is what I want. In order to create a zip file, I go to the release directory of my build.
cd /c/src/OW19/rel2/
And then clean up the $WATCOM parameter for use by Unix tools.
w=$(echo /$WATCOM | tr -d ':' | sed 's/\\/\//g')
This uses tr to delete the : character, and then sed to change backslashes to forward slashes; we use the results to set the $w parameter.
find . -newer ../bld/cc/c/cstmt.c | while read f;
do if [ -f $w/$f ];
then zip ow19a.zip $f;
fi; done
The above uses -newer to find files newer than our source file. That’s sufficient for our use. Then, we use while read f to read each line into $f, then check if $w/$f exists (whether the file exists in our installation of OpenWatcom) and if so, add it to the zip file called ow19a.zip. On first use, it’ll create the zip file, then add to it; you can download it below.
About the documentation. Ideally, we’d include documentation for the new error message, but as my build system is not configured yet to run the old wgml tool in DOSBox, nor the new wgml tool people are currently working on, I can’t test it. Furthermore, I don’t yet know if my patch will go into the OpenWatcom source tree as-is. So I have chosen not to write documentation for the error message until I’ve coordinated my efforts with the current maintainers.
Source Patch
It’s impolite not to give people a source patch as well. So let’s see if we can create one.
In cmd2 we go to the top level source directory,
cd c:\src
and rename our current working source to the patch name.
move OW19 OW19a
Note, if you get an “Access is denied” error, it means some program in the source tree is still running; just close it.
Now, we extract the pristine OpenWatcom 1.9 sources again.
7z x %homepath%\Downloads\open_watcom_1.9.0-src.zip
In cmd5 we go to the top level source directory as well,
cd /c/src
and create a patch file by comparing the two directories. Since there are lots of files that only appear in the patched source directory, that is, all the files built, and we don’t want to delete them, we pipe the difference output through grep -v to filter them out12.
diff -ur OW19/bld/cc OW19a/bld/cc/ | grep -v '^Only in' > ow19a.patch
Here, diff -ur means unified recursive difference, and grep -v means “show only those lines that don’t match those starting with Only in“, and we direct the output to a patch file, ow19a.patch.
We can now look at the file with cat, which does the same thing as Windows 10’s type command,
cat ow19a.patch
and we see
diff -ur OW19/bld/cc/c/cstmt.c OW19a/bld/cc//c/cstmt.c
--- OW19/bld/cc/c/cstmt.c 2010-02-05 14:15:38 +0800
+++ OW19a/bld/cc//c/cstmt.c 2020-04-24 21:37:08 +0800
@@ -1111,6 +1111,8 @@
case T___TRY:
if( EndTry() )
return;
+ else
+ CErr1( ERR_TRY_NEEDS_EXCEPT_OR_FINALLY );
break;
case T__EXCEPT:
case T___EXCEPT:
diff -ur OW19/bld/cc/gml/cerrs.gml OW19a/bld/cc//gml/cerrs.gml
--- OW19/bld/cc/gml/cerrs.gml 2010-03-08 14:57:34 +0800
+++ OW19a/bld/cc//gml/cerrs.gml 2020-04-24 21:33:49 +0800
@@ -2168,6 +2168,11 @@
int j = 3;
}
.eerrbad
+:MSGSYM. ERR_TRY_NEEDS_EXCEPT_OR_FINALLY
+:MSGTXT. A __try statement must be followed by either __except or __finally.
+:MSGJTXT. A __try statement must be followed by either __except or __finally.
+.np
+This error message is undocumented.
:eMSGGRP. Errs
:cmt -------------------------------------------------------------------
:MSGGRP. Info
and that’s it. Developers can use the patch program to apply this to the source of OpenWatcom 1.9, or different version if similar enough13.
Final Words
As we’ve seen from this example, fixing bugs (including null pointer reads) is not always done at the site where the pointer read happens. Here, the code generating part of the front end expects an invariant: that all __try blocks are followed by a __finally or __except. Instead of a complicated fix in the code generating part, we simply make sure this this always happens at the syntax analysis stage.
OpenWatcom 2. I haven’t tried OpenWatcom 2 more than I explained at the very start. By code inspection on GitHub, this seems to be the very bug I ran into at first, but the compiler doesn’t seem to crash, just hang. It will be up to the people on GitHub to apply this fix to their code.
Credits
The picture of the girl was drawn by the talented Ali Perez, @blackopal1227 on instagram.
Social Media
Follow me on Twitter: @myrkraverk.
Follow me on Gab: @myrkraverk.
Where I post about programming, databases, and information security.
Installation
Go to your OpenWatcom installation directory. If you don’t know it, you can probably
cd %watcom%
in Windows, and then
unzip %homepath%\downloads\ow19a.zip
or adjust to wherever you downloaded the patch file. These directions apply more or less to DOS and OS/2 as well, but you’ll have to adjust your download path, since neither of these platforms have a %homepath%.
Downloads
I’ve run this file through VirusTotal, and it has a 1/74 malicious rating, probably because of the DOS binaries. DOS binaries are notorious for false positives on VirusTotal14.
- ow19a.zip 3.5M bytes (eedcd391a4e1eb7b5a9576331dbae3ff815467c78b71a9ceff7770f7e9cb24b5)
- ow19a.patch 996 bytes (c3fef9baf990da1c1a8b256961a24bb3fc7b33e95df16cbebf4d0e58197ced26)
Footnotes
1 I’m willing to be wrong about this; please let me know if you know better.
2 I’ve not tried it. By now it’s possible the C++ compiler supports it, but the documentation hasn’t been updated. I leave it up to other people to confirm that.
3 I do not use PowerShell; I have never learned it, and I have never felt the need to learn it.
4 Which may be a pity, it might be quite good; but I don’t know.
5 Many other debuggers require the application arguments be applied to a separate run command; like gdb.
6 If you think that’s long, keep in mind I have never touched this code base before, and reading and understanding a recursive descent parser takes time. I was also lucky that I homed in on the right file at the start. Everyone who’s done code inspection knows that it often leads to unproductive time reading the wrong source files.
7 If you know of a Unix system where this has been a problem, please let me know.
8 Here I cheat slightly, this is not the error message as I first wrote it, but refined after consulting some friends.
9 I verified the file is shift-jis by converting it to utf-8 and look at it in Firefox; like this,
iconv -f shift-jis -t utf-8 cerrs.gml > cerrs.utf8
using the Mozilla build environment.
10 Technically JIS X 0201 which changes the \ symbol into Yen ¥, but close enough for our purposes.
11 Apparently I could use DOSBox for this, but it’ll have to wait a better time.
12 This is a user experienec bug in GNU diff and can be fixed.
14 In my experience with it. VirusTotal can also give false positives for things like encryption code and the like; it’s not a 100% reliable way to make sure a file is clean, there are simply too many false positives. In this case, the false positive is from Cylance which apparently uses artificial intelligence and is therefore quite likely to trigger false positives when a binary isn’t a regular application. I deemed this particular patch not worthy of a support request with them.