Tag Archives: c++

PostgreSQL: Retroactively Adding Binary I/O to a Type

Here we go through the motions af adding binary input/output functions to an already existing type. We start by introducing a program that tests binary I/O for the given types, then show how to implement the binary I/O in C and finally how to fix the PostgreSQL catalog tables for the new functionality. We also include the changes in the installation script for normal installation of the new version.
Continue reading PostgreSQL: Retroactively Adding Binary I/O to a Type

Building and Deploying 64bit FastCGI Applications in C on Windows 10

Here we go through the steps needed to build and run 64bit FastCGI applications with Visual Studio 2019.

Building the FastCGI SDK

First, get the FastCGI SDK from the Github archive. Second, open an x64 Native Tools Command Prompt for VS 2019.

Then extract the archive. 7z can do this in two steps. Here we build in C:\build but any convenient place works.

cd c:\build
7z x %homepath%\Downloads\fcgi-2.4.1-SNAP-0910052249.tar.gz
7z x fcgi-2.4.1-SNAP-0910052249.tar

We can delete the temporay .tar file now.

del fcgi-2.4.1-SNAP-0910052249.tar

Now we can try to build with the provided NT Makefile.

cd fcgi-2.4.1-SNAP-0910052249
nmake -f Makefile.nt

And we get a fatal error.

fatal error LNK1112: module machine type 'x64' conflicts with target machine type 'x86'

To figure out what’s wrong, we first take a look at the Makefile.

all: "include\fcgi_config.h"
	cd ".\libfcgi"
	$(MAKE) $(MAKEFLAGS) /NOLOGO /F libfcgi.mak     CFG=$(CFG) $@
	
	cd ".\..\cgi-fcgi"
	$(MAKE) $(MAKEFLAGS) /NOLOGO /F cgi-fcgi.mak    CFG=$(CFG) $@
	
	cd ".\..\examples"
	$(MAKE) $(MAKEFLAGS) /NOLOGO /F authorizer.mak  CFG=$(CFG) $@
	$(MAKE) $(MAKEFLAGS) /NOLOGO /F echo.mak        CFG=$(CFG) $@
	$(MAKE) $(MAKEFLAGS) /NOLOGO /F echox.mak       CFG=$(CFG) $@
	$(MAKE) $(MAKEFLAGS) /NOLOGO /F size.mak        CFG=$(CFG) $@
	$(MAKE) $(MAKEFLAGS) /NOLOGO /F echo-cpp.mak    CFG=$(CFG) $@

We see that the Makefile is calling other Makefiles, so we take a look at the first one, libfcgi.mak, and there we find this line:

LINK32_FLAGS=Ws2_32.lib /nologo /dll /pdb:none /machine:I386 /out:"$(OUTDIR)\libfcgi.dll" /implib:"$(OUTDIR)\libfcgi.lib" 

where we try to remove the /machine:I386 and rebuild.

nmake -f Makefile.nt

We get a little further this time, and fail again when building cgi-fcgi.mak. In that file we have

LINK32_FLAGS=libfcgi.lib /nologo /pdb:none /machine:IX86 /out:"$(OUTDIR)\cgi-fcgi.exe" /libpath:"..\libfcgi\Release" 

where we again remove the /machine:IX86 and try again.

We repeat this process for authorizer.mak, echo.mak, echox.mak, size.mak, and echo-cpp.mak.

Now we have a successfully built FastCGI SDK, and can try to deploy the examples.

Deploying the FastGCI Examples on Apache 2.4

Installing the FastCGI Module

Here we assume that the Apache binaries have been downloaded from the Apache Lounge. It is outside the scope of this tutorial to include how to configure Apache to start and stop with Windows. We’ll only mention that the server can be run in a console with httpd in the bin subdirectory.

For our purposes, we’ll assume Apache has been installed in C:\opt

First, extract the module in a convenient location,

7z x %homepath%\downloads\mod_fcgid-2.3.10-win64-vs16.zip

and copy the module into Apache’s root,

copy mod_fcgid-2.3.10\mod_fcgid.so C:\opt\Apache24\modules\

and then edit C:\opt\Apache24\conf\httpd.conf and add

LoadModule fcgid_module modules/mod_fcgid.so

and finally restart Apache. If you’re running it in a console, you can ctrl+c, wait for a few seconds while it shuts down, and then restart with httpd

Deploying the Examples

First, we prepare the document root for FastCGI applications. On Windows, it’s customary to put DLL files directly where the executable is, or in the system’s PATH. For our purposes, putting the libfcgi.dll file in our path seems like an overkill, so we will put it in our document root along with the executables, but we don’t want people to be able to load the DLL files. Therefore we change the document root default access to a denial.

Given

DocumentRoot "${SRVROOT}/htdocs"
<Directory "${SRVROOT}/htdocs">
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>

we change the Require line to read

    Require all denied

and then add

    <Files "*.exe">
        Options +ExecCGI
        SetHandler fcgid-script
        Require all granted
    </Files>

So our complete document root directory config is now

DocumentRoot "${SRVROOT}/htdocs"
<Directory "${SRVROOT}/htdocs">
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all denied

    <Files "*.exe">
        Options +ExecCGI
        SetHandler fcgid-script
        Require all granted
    </Files>
</Directory>

At this time we are ready to copy the executables we built earlier into our document root. In the x64 native tools command prompt we used earlier to build the SDK, copy the files into the document root we just prepared.

copy examples\echo\Release\echo.exe c:\opt\Apache24\htdocs
copy examples\echo-x\Release\echo-x.exe c:\opt\Apache24\htdocs
copy examples\echo-cpp\Release\echo-cpp.exe c:\opt\Apache24\htdocs
copy examples\size\Release\size.exe c:\opt\Apache24\htdocs
copy libfcgi\Release\libfcgi.dll c:\opt\Apache24\htdocs

And now we can try the various samples in our browser with echo.exe, echo-x.exe, echo-cpp.exe, size.exe, and finally, we make sure that downloading the DLL doesn’t work with libfcgi.dll.

If there are any failures, check the Apache error log.

Building Our Own FastCGI Application in C

We can now demonstrate how to build and deploy our very own FastCGI application, written in C.

The code is simple.

#include <fcgiapp.h>

int main( int argc, char *argv[] ) {
  FCGX_Stream *in, *out, *err;
  FCGX_ParamArray envp;

  while ( FCGX_Accept( &in, &out, &err, &envp ) >= 0 ) {
    FCGX_FPrintF( out,
                  "Content-type: text/plain\r\n"
                  "\r\n"
                  "Hello, world.\r\n" );
  }
}

This hello world example loops continuously and accepts FastCGI connections, and then just displays a plan text Hello, world message. It is just a bare bones example.

And the Makefile is also very simple. You will have to adjust the compiler path to your own installation, and the -I and -L flags. The -m64 means we’ll build a 64bit executable, the -g means we’ll include debug symbols, and the -Wall -Wextra flags tells the compiler to issue warnings when we do something dangerous or unexpected. The link flag -llibfcgi tells us to link against the DLL we just built and installed.

CC="C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\Llvm\8.0.0\bin\clang.exe"

CFLAGS=-m64 -g -Wall -Wextra -IC:\build\fcgi-2.4.1-SNAP-0910052249\include
LDFLAGS=-LC:\build\fcgi-2.4.1-SNAP-0910052249\libfcgi\Release -llibfcgi

hello.exe: hello.c
	$(CC) $(CFLAGS) hello.c -ohello.exe $(LDFLAGS)

install: hello.exe
	copy hello.exe C:\opt\Apache24\htdocs

Here, the installation target doesn’t copy the DLL file, we assume that’s been taken care of beforehand.

Happy coding FastCGI applications.

Contact

The author can be reached at johann@myrkraverk.com.

Porting a PostgreSQL Extension from Unix to Windows 10

2ndQuadrant has written before about how to build your own extension for Windows. This time we take a look at what it takes to port one written by someone else. I am building this for PostgreSQL 11 on Windows using Visual Studio 2019 and clang.

Here we are working with the hashtypes extension. Unfortunately, at the time of writing, version 0.1.5 has not been released with the support for PostgreSQL 11. So we build one from git.

git clone https://github.com/adjust/hashtypes.git

The first thing we do is to take a look at the C source files. There are only four of them, common.c, crc32.c, md5.c, and sha.c so we can be reasonably sure the porting effort will be quick.
Continue reading Porting a PostgreSQL Extension from Unix to Windows 10

Simple SHA256 Hashing with NSS in C

Here we present a program that calculates SHA256 of the string "abc" as a demonstration of how to use Mozilla’s Network Security Services library. We assume the reader has compiled a working library, or installed one through a package manager. The build instruction in this tutorial assume Windows and Visual Studio 2019 with clang.

My comments are overly verbose, and this should make it easier for beginners to follow along.
Continue reading Simple SHA256 Hashing with NSS in C

Building Mozilla NSS on Windows 10

If you’re like me you may have searched the web on how to build the Mozilla Network Security Services on Windows 10. You may have found this obsolete project on Github. You may also have found the obsolete directions for Pidgin too.

On Windows the NSS build requriements are the same as for Firefox, so go look through the build instructions for Firefox on Windows. Install Visual Studio 2019 as directed, or add the packages you need with the Visual Studio Installer. And install the Mozilla Build package in the default directory.

Now that’s done, we have one more requirement left, and that’s Gyp. For Gyp, we first need Git. If you don’t have it already, install it now. At this point I’m not certain Git comes with Visual Studio 2019. It could be, but I have a separate install anyway.

All the instructions now assume you have opened an x64 Native Tools Command Prompt for VS 2019, so open one if you don’t have already.

I prefer to keep my software I build in C:\build, so that’s what the directions will indicate.

cd c:\build
git clone https://chromium.googlesource.com/external/gyp

Gyp is a Python program, so if you don’t have it already, add Python to your path. You’ll find it in C:\mozilla-build\python and also add gyp to your path, like so

set path=%path%;C:\mozilla-build\python;C:\build\gyp

Download and extract nss-3.45-with-nspr-4.21.tar.gz By the time you read this, that particular download might be obsolete, so please adjust as needed.

I use 7z for my command line examples.

cd c:\build
7z x %homepath%\Downloads\nss-3.45-with-nspr-4.21.tar.gz

That only creates the tar file, so we need to run 7z again.

7z x nss-3.45-with-nspr-4.21.tar

You may need to add vswhere to your path, and you can do that now with

set path=%path%;%ProgramFiles(x86)%\Microsoft Visual Studio\Installer

At this point we’ll need an msys shell. So still in your x64 Native Tools Command Prompt for VS 2019,

cd C:\mozilla-build\msys
msys

In msys, cd to the place where you extracted NSS.

cd /c/build/nss-3.45/nss

And build with

./build.sh

And you should get a nicely compiled debug build in C:\build\nss-3.45\dist. Happy hacking with NSS.

Contact

The author can be reached at johann@myrkraverk.com.

The sizeof Operator in C and C++

Textbooks rarely make good use of the sizeof operator in C (and C++). The syntax is

sizeof ( type )

and

sizeof variable

That is, in order to get the size of a variable, including an array, the parenthesis are not necessary. I personally find them a visual distraction, unless of course the operand is a type, like sizeof ( int ).

Let’s look at a concrete example of using sizeof without parenthesis. In this example, we’re preparing an error message for display on screen.

char buffer[ 1024 ]; /* Arbitrary size of the buffer. */
snprintf( buffer, sizeof buffer, "Unable to initialize SDL: %s", SDL_GetError() );

Because we’re using sizeof buffer in the snprintf() call we don’t have to worry about mistakes, or out of sync constants, and we don’t have to #define BUFFER_SIZE 1024 to use the same size in both definition of the buffer and call to snprintf().

Note to Windows programmers. The snprintf() function isn’t documented to always terminate the string with zero until Visual Studio 2015 and Windows 10. Programmers on the Windows platform might want to add an explicit zero termination to account for older compilers and systems. That can be done with buffer[ sizeof buffer - 1 ] = '\0'.

The trick here is that we defined buffer as an array. If we had instead used malloc() to allocate the buffer, we would have to add the size of the buffer explicitly, like so,

char *buffer = malloc( 1024 );
if ( !buffer ) exit( 1 );
snprintf( buffer, 1024, "Unable to initialize SDL: %s", SDL_GetError() );

and we would have to explicitly check the return value from malloc() as is written about at length in why it is important to check what the malloc function returned. If we had instead used sizeof buffer here, we’d have gotten 4 on a 32bit system, and 8 on a 64bit system — which is totally not the value we need.

The snprintf() function returns how many characters have been or would be printed. In our case we don’t care if the message the user receives gets truncated, so we don’t check its return value.

It is worth noting that when a character literal is used, it matters whether the literal is declared as a pointer or array. That is, given

  char *foo   = "Error string.",
        bar[] = "Another error string.";

then sizeof foo will give us 4 on 32bit system, 8 on a 64bit system, while sizeof bar is 22 or the length of the string including the terminating zero byte.

Literals

The sizeof operator applies to literals also without parenthesis, and it’s instructive to test it on some literal combinations on a given system. For example, this program

#include <stdio.h>

int main( int argc, char *argv[] )
{
  printf( "sizeof 0 = %zu\n",       sizeof 0 );     // int
  printf( "sizeof 0l = %zu\n",      sizeof 0l );    // long
  printf( "sizeof 0ll = %zu\n",     sizeof 0ll );   // long long
  printf( "sizeof NULL = %zu\n",    sizeof NULL );  // pointer
  printf( "sizeof 0.0 = %zu\n",     sizeof 0.0 );   // double
  printf( "sizeof 0.0f = %zu\n",    sizeof 0.0f );  // float
  printf( "sizeof \"foo\" = %zu\n", sizeof "foo" ); // string size including zero terminator
  return 0;
}

run an a 64bit Linux and compiled with clang, gives

sizeof 0 = 4
sizeof 0l = 8
sizeof 0ll = 8
sizeof NULL = 8
sizeof 0.0 = 8
sizeof 0.0f = 4
sizeof "foo" = 4

and none of that should be surprising.

Contact

The author can be reached at johann@myrkraverk.com.

Updates

Added the section on literals.

Hello World with SDL2

Ok, you’ve gone through the texture tutorial for Grafx2 and are eager to try out your skills with SDL2 programming. Look no further, we’re going to build a very basic “hello, world” application that should get you started writing great games.

Hello World

This tutorial is in plain C and not C++. This is in direct contradiction with most examples out there, and is helpful to showcase SDL usage, rather that game design of any particular author.

This tutorial is not meant to replace what is available on the Lazy Foo’ Productions web site, rather to complement it.

The code is explicitly WTFPL for maximum freedom. See the WTFPL website for further details.

Continue reading Hello World with SDL2

OS/2 Sequence Driver

Introduction

Programmers often have a need for a unique identifier for various reasons. Sometimes people end up using databases for a simple reliable counter, when there’s no other need for a database.

This is overkill. There’s no need to depend on something like Postgres just because we need a simple counter. But implementing a reliable counter can be a daunting task. Particularly if there is more than one application using the same counter.

This is a service that could very well be implemented in the operating system. When uniqueness and perhaps order is all that’s required, it’s perfectly all right for application foo and bar to use the same counter. All it means, is that when foo requests a new value, the counter may have been incremented by bar.

We can call this a non-decreasing counter. For an individual application foo, the results could be 1, 2, and 5; when application bar has 3, 4 and 6.

Concept

Here we present a simple software driver that creates a device that can be opened and read like a regular file, but each read results in a new value from the counter. We shall call it dev$seq$ so that it will be unlikely to conflict with regular file names.

As a proof of concept this driver lacks certain features that are required in a real world application. First, it’s only 16 bits so it’ll wrap around at 65,535 and become zero. Second, there’s no way to save its value to the file system; it always starts at zero upon every reboot. Third, there’s no backup procedure that can be applied.

Continue reading OS/2 Sequence Driver

Building OpenSSL with OpenWatcom on ArcaOS: Second Porting Effort

I’ve written before about my OpenSSL port to OS/2. I’m doing this with OpenWatcom 1.9 on the ArcaOS flavour of OS/2.

Building with no Assembler

First, we copy OS2-OW.cmd to OS2-OW-NOASM.cmd and then make the new file run perl Configure OS2-OW no-asm and perl util\mk1mf.pl no-asm OS2-OW > OS2-OW.mak.

Then it’s just a matter of building the make file with OS2\OS2-OW-NOAM.cmd and building with wmake -u -c -f OS2-OW.mak.

This builds the libraries ssl.lib and crypto.lib plus a few applications. Some of them work.

apps/ca.c

This file fails to build with the error message .\apps\ca.c(86): Error! E1055: Unable to open 'sys/file.h' and we fix that by adding

&& !defined(OPENSSL_SYS_OS2)

on line 85 where it’s including sys/file.h and now this application builds also.

The Heisenbug

Now that we have crypto.lib and ssl.lib it’s time to try out OpenSSL with a simple example. Copying some code from the internet, and trying it out, we get a mysterious crash inside

SSL_CTX_new( method )

which causes the example program to terminate.

In order to debug this problem, we start by changing the optimization parameters and replace them with debugging symbols. In util/pl/OS2-OW.pl we make

$cflags="-DL_ENDIAN -d2 ";

in order for OpenSSL to be built with debugging symbols instead of optimizations.

Then we just delete everything in the out/ and tmp/ directories and build all over again with OS2-OW-NOASM.cmd and wmake -u -c -f OS2-OW.mak.

Lo and behold, there is no crash when we link with the new .lib files. This is definitely a Heisenbug.

Final Words

We now have a somewhat functioning OpenSSL port and can make simple applications such as hello, world work, albeit with a debug build, but more research is needed before the port can be considered finished. Particularly because there is a Heisenbug in the optimized version, which could be a code generation bug in OpenWatcom, or a symptom of something broken inside OpenSSL.

Do you need professional help with OS/2? Write to johann@myrkraverk.com.

Polymorphism in Plain C

Here we go through the steps required to implement polymorphic interfaces in plain C. We use function pointers for this task, hidden behind generic functions we define for the interface itself.

To demonstrate the technique, we implement a simple queue of string pointers. This entry is about the generic interface so some deficiencies and possibly bugs in the actual implementation may pass us by. Please write the author or comment on the post if you spot errors in the implementation.

First we define the interface we’re going to use. We start off by defining a struct with the function pointers we need.

struct queue {

  void *secret;

  void (* enqueue)( struct queue *, char * );
  char * (*dequeue)( struct queue * );
  bool (*empty)( struct queue * );
  struct queue * (* delete)( struct queue * );
};

The void *secret is what we use in the implementation to keep track of our secret data structure. The rest are the function pointers we need to define for each implementation.

Here we use direct function pointers for all of the functions. We could also put the pointers into a separate struct for easier sharing, or at least smaller concrete objects, but we leave that optimization as an exercise for the dedicated reader.
Continue reading Polymorphism in Plain C