Paul Johnson
July 11, 2001

Build Swarm in Cygnus?

You may recall the question: Can I build swarm in the current Cygwin
environment?  I have Cygwin and Swarm seems to duplicate many
programs, and also Cygwin and Swarm compete to knock each other's
mount's out of place.

I've gone round and round with Marcus, who has been most eager to
provide advice, but in the end, I think the conclusion is "you should
install the Swarm windows distribution and within that you can rebuild
swarm from source if you want to, but the cygwin distribution is not
very cooperative." 

Maybe you are good at windows and want to try this yourself. Here is
a digest of my experience.

Install cygwin. Install all their packages.

Install xfree86 version 4.1, also from cygnus.  Dump all their tar.bz2
files into c:\cygwin\ and they will endup in c:\cygwin\usr\X11r6.

Here is the first big horrible fact. The Cygwin version of gcc has
objective-C support removed.  Dang.  The answer appeared easy, since I
was able to build gcc3.0 without any fuss and it did work to compile
ordinary C programs. 

You first think you will get gcc-2.95.3 from the cygwin people and
enable objective C in their makefile, but they have completely
eliminated the objc code, and there's no hope there. You can get the
source for gcc from the GCC team, but I could not get that to build
correctly. There are numerous known problems caused by a change in the
cygnus environment. That build fails because it can't find
<windows.h>. I was able to make the build complete by hand editing the
Makefiles to make sure that the INCLUDES for gcc had /usr/include and
/usr/include/w32api. However, it is not a success because, after
installing, the gcc was not able to create executables.

So then turn to gcc-3.0. Download gcc-3.0 source, it is easy to build.
Untar it wherever, apply the objective-c patch Marcus put on the email
list (I can send it to you if you want-- ask for
Marcus_combo_objc-gcc3.0.patch),then

 mkdir buildgcc
 cd buildgcc
 ../gcc-3.0/configure --prefix=/usr
 make bootstrap

Pause here, then use the cygwin setup tool to remove the gcc 2.95 they
gave you, then:

 make install

So, assuming you get past the gcc problem, then you need:

1.  replace the automake and autoconf that are disributed with cygwin.
Get the new ones from ftp.swarm.org (look under src/developer), and
then do the easy thing for each "./configure --prefix=usr" "make" and
"make install".

2. download/install the libtool source code. That is an easy build,
put it in the same dir as automake and autoconf.  I first tried the
libtool in the swarm ftp and later installed libtool 1.4 from the
libtool site.

3. You need java support. Java is not optional, it is needed to
compile the swarm base at the current time (maybe not forever).  I
chose to download the windows version from www.javasoft.com and
install into c:\jdk1.3.1.

You eventually run into a problem with definitions for 64 bit
integers. In this file: c:\jdk1.3.1\include\win32\jni_md.h you find
this:

typedef __int64 jlong;

And it caused me a hell of a time!  It assumes you have __int64
defined, and due to various problems, you don't. Lacking patience to
do it with the swarm makefiles and such, I TEMPORARILY fixed the
problem by changing it to this:

typedef long long jlong;

If swarm ever compiles and works, then I/you can figure out how to fix
this correctly. This all links back to the swarm setting in
swarmconfig.h, where both INT64 and JNI_H_NEEDS_INT64 are defined.
More on that later.

4. You need BLT for swarm. Building from scratch is a pain because it
requires source for tcl/tk and some complicated maneuvers. There are
binaries available, but installing them is tricky because of the
reason described in the next paragraph.  If you want to build from
scratch, the BLT tarball (not the windows install executable) has
detailed building instructions just for windows.

Now, a soliloquy: 
There is a big weirdness of cygwin install
directories. Type "mount" and you will see it. Note They use /usr as
the prefix for all installs, but for reasons I cannot understand, they
move all those files out of c:\cygwin\usr\lib and c:\cygwin\usr\bin
into c:\cygwin\usr\lib int c:\cygwin\bin and c:\cygwin\lib, and then
the mount setup is used to make everything in their bin directory
appear as /usr/bin. The MONSTER problem this causes is that, if you
untar compiled programs from other places, then their files do not end
up where they need to be.  If you want a file to show up as
/usr/bin/myProgram, you need to dump it into c:\cygwin\bin.
Unfortunately, if you untar a program that has paths in it into
c:\cygwin, it will end up in c:\cygwin\usr\bin\, and the system won't
find it, because the system is looking in c:\cygwin\bin when programs
says /usr/bin. See?  So you have to double-check installs and move
files from c:\cygwin\usr\bin into c:\cygwin\bin and from
c:\cygwin\usr\lib into c:\cygwin\lib.

I wonder if there is a good long term fix? Until then....

Now, back to BLT.  The easiest road is to install the blt8.0 binary
from http://www.tcltk.com/blt. Tell it to install into c:\cygwin\usr,
and after it drops files into c:\cygwin\usr\lib and c:\cygwin\usr\bin,
you have to move them into c:\cygwin\bin and c:\cygwin\lib. Another
problem is that the blt installs a library file with a wrong name, it
installs in the lib dir BLT24.lib, and in order to make swarm compile,
you have to copy that file to libBLT24.a in the lib directory.  I
never understood why that version of blt did that.  We ran into it on
unix systems too a couple of years ago.

Now read through the rest before you start typing, because there are gotchas:

Untar swarm distribution wherever you want, and follow along like this.

 mkdir buildSwarm 
  cd buildSwarm 
  EMACS=c:/emacs-20.7/bin/emacs.exe
              ../swarm-2001-06-27/configure --prefix=/Swarm-20010627
                 --with-default-dir=/usr --with-tclscriptdir=/usr/share/tcl8.0
                 --with-jdkdir=/cygdrive/c/jdk1.3.1 make

(if you want to try to make a version that does the objective-c
message speedup you can do it with "make
EXTRACPPFLAGS=-DMETHOD_FUNCTIONS". There is another swarm patch you
can apply in his email of July 6, 2001)


When swarm builds, you will encounter various troubles.


There is a problem compiling the file misc.c in the swarm distro.  The
make fails with errors about xmalloc not found.  Had to diddle the end
of src/libobjc/misc.c to make sure the WIN32 part was getting
processed.  If the OSTYPE and other environment and swarm configure
stuff was just right (see below), maybe this would not be a problem

Before I fixed the jdk __int64 thing, my swarmconfig.h was declaring
INT64.  That declaration conflicted with the file
c:\cygwin\usr\include\w32api\basetsd.h.  You will see the conflict as
a crash in the build, so check your swarmconfig.h in the  build directory.

Cygnus changed the OSTYPE in the environment.  OSTYPE in environment
is now cygwin, not cygwin32, as it was. So you change cygwin32 to
cygwin in the swarm source dir etc/Makefile.protocol in this line
(line 8 on my system):

ifeq ($(OSTYPE),cygwin)
  dos_abs_top_builddir := $(shell cygpath -w $(abs_top_builddir))
  dos_abs_builddir := $(shell cygpath -w $(abs_builddir))
  dos_swarm_srcdir = $(shell cygpath -w $(swarm_srcdir))
.

Maybe if you set your OSTYPE to cygwin32 in the environment, this 
would be OK as well.

The dllwrap program that cygnus distribues does not work to correctly
build libraries. It is simply no good. Marcus had discovered that a
while ago, and the Swarm distro has a fixed-up version.  Replace that
with dllwrap.exe from swarm2.1.1.  If you don't do that, it builds bad
dlls and then about 1/2 through the swarm build, you start seeing a
ton of errors about objc_msg_lookup not defined.

So, making all these changes, the swarm build crashes like so, after
it goes very very far:

gcc: unrecognized option -R

it has tried to use gcc with a flag -R /usr/lib and that's no good.

The "fix" for that is to go into Makefile.common and
delete the -R references.  I guess, at least for a short term fix.

After that, the build arrives at the same place, with this ignominious conclusion:


make[3]: Entering directory `/ddrive/home/pjohnson/src/buildSwarm010706/src'
gcc -g -c ../../swarm-2001-07-06/src/swarm.dll.c
/usr/bin/dllwrap --export-all-symbols --output-def swarm.def --implib libswarmdll.a --driver-name gcc -o swarm.dll swarm.dll.o space/*.lo analysis/*.lo simtoolsgui/*.lo simtools/*.lo random/*.lo tkobjc/*.lo tclobjc/*.lo objectbase/*.lo activity/*.lo defobj/*.lo collections/*.lo misc/*.lo SwarmEnvironment.lo -L../libobjc   -L/usr/lib -L/usr/lib -L/usr/lib   -L/usr/X11R6/lib  -L/usr/X11R6/lib -lobjcdll -lBLT24 -ltk80 -ltcl80 -lXpm -lpng  -lz ../avcall/avcall-i386-msvc.lo ../avcall/avcall.lo -luser32 -lgdi32 -lm 
Warning: no export definition file provided
dllwrap will create one, but may not be what you want
space/Ca2d.lo: In function `OBJC_METH_VAR_NAME_60':
//d/home/pjohnson/src/buildSwarm010706/src/space/../../../swarm-2001-07-06/src/space/Ca2d.m(.data+0x57c): undefined reference to `__objc_class_name_Protocol'
space/ConwayLife2d.lo: In function `OBJC_METH_VAR_NAME_35':
//d/home/pjohnson/src/buildSwarm010706/src/space/../../../swarm-2001-07-06/src/space/ConwayLife2d.m(.data+0x5bc): undefined reference to `__objc_class_name_Protocol'
space/DblBuffer2d.lo: In function `OBJC_METH_VAR_NAME_27':
//d/home/pjohnson/src/buildSwarm010706/src/space/../../../swarm-2001-07-06/src/space/DblBuffer2d.m(.data+0x55c): undefined reference to `__objc_class_name_Protocol'
space/Diffuse2d.lo: In function `OBJC_METH_VAR_NAME_19':
//d/home/pjohnson/src/buildSwarm010706/src/space/../../../swarm-2001-07-06/src/space/Diffuse2d.m(.data+0x61c): undefined reference to `__objc_class_name_Protocol'
space/Discrete2d.lo: In function `i_Discrete2d__hdf5In_':
//d/home/pjohnson/src/buildSwarm010706/src/space/../../../swarm-2001-07-06/src/space/Discrete2d.m:174: undefined reference to `__objc_class_name_Protocol'
space/Grid2d.lo(.data+0x4dc)://d/home/pjohnson/src/buildSwarm010706/src/space/../../../swarm-2001-07-06/src/space/Grid2d.m: more undefined references to `__objc_class_name_Protocol' follow
tkobjc/internal.lo: In function `tkobjc_animate_message':
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:225: undefined reference to `XFlush'
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:282: undefined reference to `XCreateWindow'
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:287: undefined reference to `XPutImage'
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:386: undefined reference to `XFlush'
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:265: undefined reference to `XCopyColormapAndFree'
tkobjc/internal.lo: In function `tkobjc_raster_drawPoint':
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:566: undefined reference to `XDrawPoint'
tkobjc/internal.lo: In function `tkobjc_setColor':
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:640: undefined reference to `XCopyColormapAndFree'
tkobjc/internal.lo: In function `x_get_parent_window':
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:686: undefined reference to `XFree'
tkobjc/internal.lo: In function `x_get_wm_child':
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:708: undefined reference to `XFree'
tkobjc/internal.lo: In function `tkobjc_raster_createPixmap':
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:829: undefined reference to `XCreatePixmap'
tkobjc/internal.lo: In function `tkobjc_raster_dropOldPixmap':
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:957: undefined reference to `XFreePixmap'
tkobjc/internal.lo: In function `check_for_overlaps':
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:1202: undefined reference to `XFree'
tkobjc/internal.lo: In function `tkobjc_pixmap_create_from_widget':
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:1308: undefined reference to `XReparentWindow'
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:1329: undefined reference to `XReparentWindow'
tkobjc/internal.lo: In function `tkobjc_pixmap_update_raster':
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:1406: undefined reference to `XFreePixmap'
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:1411: undefined reference to `XFreePixmap'
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:1451: undefined reference to `XCopyColormapAndFree'
tkobjc/internal.lo: In function `tkobjc_pixmap_drop':
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:1795: undefined reference to `XFreePixmap'
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:1797: undefined reference to `XFreePixmap'
tkobjc/internal.lo: In function `tkobjc_raster_flush':
//d/home/pjohnson/src/buildSwarm010706/src/tkobjc/../../../swarm-2001-07-06/src/tkobjc/internal.m:865: undefined reference to `XFlush'
tclobjc/HashTable.lo: In function `i_HashTable__init':
//d/home/pjohnson/src/buildSwarm010706/src/tclobjc/../../../swarm-2001-07-06/src/tclobjc/HashTable.m:140: undefined reference to `__objc_class_name_Object'
tclobjc/List.lo: In function `i_TclObjcList__isEqual_':
//d/home/pjohnson/src/buildSwarm010706/src/tclobjc/../../../swarm-2001-07-06/src/tclobjc/List.m:143: undefined reference to `__objc_class_name_Object'
tclobjc/TclInterp.lo: In function `fix_tcl_path':
//d/home/pjohnson/src/buildSwarm010706/src/tclobjc/../../../swarm-2001-07-06/src/tclobjc/TclInterp.m:174: undefined reference to `__objc_class_name_Object'
objectbase/ActivityControl.lo: In function `i_ActivityControl___setup_ProbeMap':
//d/home/pjohnson/src/buildSwarm010706/src/objectbase/../../../swarm-2001-07-06/src/objectbase/ActivityControl.m:184: undefined reference to `__objc_class_name_Protocol'
objectbase/CompleteProbeMap.lo: In function `OBJC_CLASS_NAME_8':
//d/home/pjohnson/src/buildSwarm010706/src/objectbase/../../../swarm-2001-07-06/src/objectbase/CompleteProbeMap.m(.data+0x4dc): undefined reference to `__objc_class_name_Protocol'
objectbase/CompleteVarMap.lo: In function `OBJC_CLASS_NAME_10':
//d/home/pjohnson/src/buildSwarm010706/src/objectbase/../../../swarm-2001-07-06/src/objectbase/CompleteVarMap.m(.data+0x4bc): undefined reference to `__objc_class_name_Protocol'
objectbase/CustomProbeMap.lo: In function `OBJC_METH_VAR_NAME_6':
//d/home/pjohnson/src/buildSwarm010706/src/objectbase/../../../swarm-2001-07-06/src/objectbase/CustomProbeMap.m(.data+0x520): undefined reference to `__objc_class_name_Protocol'
objectbase/DefaultProbeMap.lo: In function `OBJC_METH_VAR_NAME_13':
//d/home/pjohnson/src/buildSwarm010706/src/objectbase/../../../swarm-2001-07-06/src/objectbase/DefaultProbeMap.m(.data+0x47c): undefined reference to `__objc_class_name_Protocol'
objectbase/EmptyProbeMap.lo(.data+0x4fc)://d/home/pjohnson/src/buildSwarm010706/src/objectbase/../../../swarm-2001-07-06/src/objectbase/EmptyProbeMap.m: more undefined references to `__objc_class_name_Protocol' follow
defobj/DefClass.lo: In function `i_CreatedClass_s__createEnd':
//d/home/pjohnson/src/buildSwarm010706/src/defobj/../../../swarm-2001-07-06/src/defobj/DefClass.m:153: undefined reference to `__objc_class_name_Object'
defobj/DefObject.lo: In function `i_Object_s__addRef_withArgument_':
//d/home/pjohnson/src/buildSwarm010706/src/defobj/../../../swarm-2001-07-06/src/defobj/DefObject.m:363: undefined reference to `__objc_class_name_Object'
//d/home/pjohnson/src/buildSwarm010706/src/defobj/../../../swarm-2001-07-06/src/defobj/DefObject.m:362: undefined reference to `__objc_class_name_Protocol'
defobj/HDF5Archiver.lo: In function `i_HDF5Archiver_c__ensureApp_':
//d/home/pjohnson/src/buildSwarm010706/src/defobj/../../../swarm-2001-07-06/src/defobj/HDF5Archiver.m:105: undefined reference to `__objc_class_name_Protocol'
defobj/HDF5Object.lo: In function `GLOBAL__I__i_HDF5CompoundType_c___I_Creating.1':
//d/home/pjohnson/src/buildSwarm010706/src/defobj/../../../swarm-2001-07-06/src/defobj/HDF5Object.m:2198: undefined reference to `__objc_class_name_Protocol'
defobj/JavaCollection.lo: In function `i_JavaCollection__addLast_':
//d/home/pjohnson/src/buildSwarm010706/src/defobj/../../../swarm-2001-07-06/src/defobj/JavaCollection.m:79: undefined reference to `__objc_class_name_Protocol'
defobj/JavaProxy.lo: In function `i_JavaProxy__createJavaCounterpart_':
//d/home/pjohnson/src/buildSwarm010706/src/defobj/../../../swarm-2001-07-06/src/defobj/JavaProxy.m:70: undefined reference to `__objc_class_name_Protocol'
defobj/LispArchiver.lo://d/home/pjohnson/src/buildSwarm010706/src/defobj/../../../swarm-2001-07-06/src/defobj/LispArchiver.m:156: more undefined references to `__objc_class_name_Protocol' follow
defobj/Program.lo: In function `initModules':
//d/home/pjohnson/src/buildSwarm010706/src/defobj/../../../swarm-2001-07-06/src/defobj/Program.m:174: undefined reference to `__objc_class_name_Object'
defobj/Symbol.lo: In function `i_Error_c__raiseEvent':
//d/home/pjohnson/src/buildSwarm010706/src/defobj/../../../swarm-2001-07-06/src/defobj/Symbol.m:160: undefined reference to `__objc_class_name_Protocol'
defobj/defobj.lo: In function `lispInBoolean':
//d/home/pjohnson/src/buildSwarm010706/src/defobj/../../../swarm-2001-07-06/src/defobj/defobj.m:197: undefined reference to `__objc_class_name_Protocol'
collections/Array.lo: In function `i_Array_c__setCount_':
//d/home/pjohnson/src/buildSwarm010706/src/collections/../../../swarm-2001-07-06/src/collections/Array.m:238: undefined reference to `__objc_class_name_Protocol'
collections/Collection.lo: In function `i_Collection_any__describeForEach_':
//d/home/pjohnson/src/buildSwarm010706/src/collections/../../../swarm-2001-07-06/src/collections/Collection.m:289: undefined reference to `__objc_class_name_Protocol'
collections/InputStream.lo: In function `i_InputStream_c__getExpr':
//d/home/pjohnson/src/buildSwarm010706/src/collections/../../../swarm-2001-07-06/src/collections/InputStream.m:350: undefined reference to `__objc_class_name_Protocol'
collections/List.lo://d/home/pjohnson/src/buildSwarm010706/src/collections/../../../swarm-2001-07-06/src/collections/List.m:138: more undefined references to `__objc_class_name_Protocol' follow
collect2: ld returned 1 exit status
/usr/bin/dllwrap: gcc exited with status 1
make[3]: *** [swarm.dll] Error 1