swig runs forever


03-10-2005 12:36:58

I'm just trying to get up and running with Ogre and PyOgre on Mac OS X. I've got the latest Ogre release up and running, now time for PyOgre. The only packaged releases I could find were old ones only for Windows, so I grabbed the trunk from svn and am trying to get things going. The readme file for Mac OS X has sort of a walkthrough, but I get hung up on the first step:

swig -c++ -python -modern -I/Library/Frameworks/Ogre.framework/Headers \
> pyogre/ogre/ogre.i
pyogre/ogre/OgreStringVector.i:9: Warning(404): Template 'SharedPtr<std::vector<Ogre::String > >' was already wrapped,
pyogre/ogre/OgreStringVector.i:9: Warning(404): previous wrap of 'SharedPtr<std::vector<Ogre::String > >'.
pyogre/ogre/OgreIteratorWrappers.i:7: Warning(404): Template 'MapIterator<std::multimap<Ogre::String,Ogre::String > >' was already wrapped,
pyogre/ogre/OgreIteratorWrappers.i:7: Warning(404): previous wrap of 'MapIterator<std::multimap<Ogre::String,Ogre::String > >'.

Those warnings popped up during the first few seconds, and then it just sat there. It was using CPU, but it just sat there for an hour before I got fed up and killed it.

Should it really take that long to do what it needs to do? Should I have let it keep on going? This is on a 1.33MHz iBook, with swig 1.3.25


03-10-2005 12:55:12

I don't know exactly why but swig 1.3.25 is sucking a lot more memory than 1.3.24 and is much much slower. You should have just let it run.. :(


03-10-2005 13:07:53

OK, thanks for the tip. I tried re-running with -v, but it didn't tell me much more (it stops forever after "Generating Wrappers").

So later when I'm not actually sitting at the machine (cause boy howdy, swig really hogs the CPU!) I'll let it run until it's done. Thanks again!


04-10-2005 14:15:33

I have never tried it on OSX (in fact i have never even seen OSX up close), but on windows it takes more than 10 minutes (wall clock) to run the swig step.

If you guys say older versions are faster, i'll switch in a heartbeat.


04-10-2005 18:30:42

Much faster (like 1-2 minutes vs about 8 on my laptop.)


04-10-2005 22:53:13

Try seventy (70)! Seventy minutes on my freshly rebooted iBook with (essentially) no other applications running. I'd *love* to have it run in just 10 minutes! Well, at least it's not something that needs to occur frequently...


04-10-2005 23:25:52

... unless you are fiddling with the .i files, and that is unfortunately a prerequisite for any kind of development on pyOgre itself.


04-10-2005 23:49:13

Yep, but I use a trick. When trying to fix a bug I comment out from ogre.i almost everything that I am not supposed to call from Python. Even if you wrap just a couple of classes Ogre will still work; you just won't be able to hook into it. My usual development cycle, after isolating something to work on is about 20-30 seconds to go from .i to _ogre.so.


70 minutes is a lot. Maybe swig on mac has even more problems than swig on win or linux. :(


05-10-2005 03:10:10

On my computer running Windows 98SE 1.5MHz / 512 megs of RAM it also took about 70 minutes to run. Same machine - linux 10 minutes.


05-10-2005 19:58:24

Never had to tweak the .i file myself. Takes a few minutes on my meaty Linux (AMD64) desktop.


05-10-2005 20:06:14

Could it be that directors are causing the slow build time? I know the swig documentation recommends you only enable directors for select classes, because of the amount of bloat they cause.


05-10-2005 20:25:36

It is on the TODO list (in fact I didn't even looked at the directors, Clay did the work..)


09-10-2005 21:04:48

I fixed this problem today (as of revision 196). The problem was related to OgreConfigFile.i, but I'm not sure what exactly causes the problem. In the original code you could comment out %include OgreConfigFile.i (in Ogre.i) and this would fix the problem (but you would not have a ConfigFile class). To get around all of this ugliness I've reimplemented the ConfigFile class in Python (you can see it in OgreConfigFile.i). It does not work as it used to, and it may not be totally error free, but it does the job for now.

You can use the old method of calling config file options:
config = ogre.ConfigFile()
for sectionName, mm in config.getSectionIterator():
if mm:
for key, path in mm.items():
ogre.ResourceGroupManager.getSingleton().addResourceLocation(path, key, sectionName)

Or you can use the new method, which is much cleaner:
config = ogre.ConfigFile()
for section, key, path in config.values:
ogre.ResourceGroupManager.getSingleton().addResourceLocation(path, key, section)

Either will work, but I would like to mark the first method as depreciated.


10-10-2005 00:14:53

It seems like there's something wrong. Our game now crashes when using ConfigFile to parse the leveldata (which is game specific, i.e. it doesn't have anything to do with ogre, we just use it as a way to specify the levels).

This is the code:

self.spawnPlaces = []

### Load level data from file
### Start with creating a config object and loading
### the proper config file
config = ogre.ConfigFile()
### This creates a.. list?.. of mappings between section names
### and maps (dictionaries)
### So we go through each section
for sectionName, mm in config.getSectionIterator():
### Start by checking if the dict is valid
if mm:
print "**", sectionName, "**"
#for key, path in mm.items():
# print "k", key, "\t\tp",path

### First check if the name of the section is "General"
### This section sets up stuff like the name of the level
### and the starting position of the player's spaceship
if sectionName == "General":
#self.resolution = mm["Resolution"]
self.levelName = mm["Name"]
self.waterLevel = float(mm["WaterLevel"])
startPos = (int(mm["StartPositionX"]),
wallSize = int(mm["InitialWallSize"])

### Then go through each spawn described in the file
### For now, just put it there
### TODO: Check for time delays as well
elif "Spawn" in sectionName:
x, z = int(mm["StartPositionX"]), int(mm["StartPositionZ"])
spawnType = mm["SpawnType"]
#self.tiles[x][z] = spawnType
spawn = SpawnPlace((x,0,z), spawnType, timeUntilSpawn)

### Then go through static terrain stuff described in the file
elif "Static" in sectionName:
x, z = int(mm["StartPositionX"]), int(mm["StartPositionZ"])
spawnType = mm["SpawnType"]
timeUntilSpawn = 0.1
spawn = SpawnPlace((x,0,z), spawnType, timeUntilSpawn)
self.tiles[x][z] = -1000
print "LEVEL: Level scanned."

It crashes before it reaches the last print. I'll look at it more tomorrow in case you don't see the problem instantly.


10-10-2005 01:20:13

Clay, you rule!

From 20 minutes to 20 seconds! I love you :)


10-10-2005 02:05:16

Yikes. I didn't realize someone was using it for something other than ogre configuration files. I can get to work on rewriting it if you want, but I highly recommend using the ConfigParser module from python instead: http://www.oreilly.de/catalog/pythonsl/chapter/ch05.html#t8.

This will only work if you do not actually use the multimap features. That is, this will not work:
Value1 = something
Value1 = Something Else
Value2 = other

Because the Python module does not support multiple values to the same key. (Also note that semicolons are the comment delimiter, not #.) Take a look at Python's ConfigParser class and see if it will do what you want it to. If not I'll try start work on making the new ConfigFile compatible, but I won't be able to get started until tomorrow at the earliest.


10-10-2005 19:36:53

Really nice job Clay, now I can swig/compile pyogre/cegui straight from Codeblocks in a matter of minutes.