Importing/Exporting meshes

I'm fairly new to Python-Ogre but not to python or Ogre if that makes sense . I am developing a game in C++ using Ogre3D but I'd like to use Python-Ogre to create some tools to help with development.

Basically I want to create a texture atlas tool. The basic procedure in my mind is as follows:

- Read in a number of .mesh files
- Parse the materials and get a list of all the textures
- Assemble all the textures into 1 large texture (probably using PIL for that)
- Create a new material definition using this new texture
- Assign this new material to all the meshes
- Adjust the mesh UV coordinates so that they match the new large texture
- Write out all the meshes back to .mesh files

But I'm stuck on the very first point

My first idea was to use the Ogre::MeshSerializer directly like this:

ser = ogre.MeshSerializer()
ser.importMesh()

But importMesh takes a DataStreamPtr as the first parameter. This means opening a file using either the C way (FILE*) or C++ way (std::ifstream) and passing it to Ogre::FileHandleDataStream or Ogre::FileStreamDataStream respectively. But I can't see how I can create a DataStream from the python way of reading files (i.e. file = open(...)).

Ok so I thought about a different approach:

root = ogre.Root()
ogre.ResourceGroupManager.getSingleton().addResourceLocation('.', 'FileSystem')
mesh = ogre.MeshManager.getSingleton().load("clay.mesh", ogre.ResourceGroupManager.DEFAULT_RESOURCE_GROUP_NAME)


This runs, but it crashes right after I see this message on as output:

Mesh: Loading clay.mesh

I'm guessing its because I didn't initialise all the Ogre stuff, render window, camera and all that stuff. But I'd kind of like to avoid doing that mainly to keep my code is minimal as possible but also to decrease run-time. Does anybody know what the minimum I need to initialise in order for this to work without crashing?

Hmm.. well I've got something working but I wouldn't exactly call it minimal. I've based it off the Demo_Basic.py example. I'm still required to initialise the root, plugins and create a render window in order to stop it from crashing :/ The main anoying thing is that it needs a "plugins.cfg" file in the same directory as the script and the .dll plugin files, although I could probably reduce that to just 2 files (the plugins.cfg file with the opengl rendersystem). But it all seems a bit unnecessary as I don't want to render anything. I'm considering converting the mesh to xml and just writing a python script to do the necessary operations on the xml file then converting it back to .mesh with the xml converter tools. But I'd like to keep that as a last resort.

I was wondering, would it be possible to derive a class from the virtual base class "DataStream" directly in Python? Then I could overload all the required member functions and base it on the python way of reading files. Not that many functions to deal with actually:

template<typename T>
DataStream & operator>> (T &val)
virtual size_t read (void *buf, size_t count)=0
Read the requisite number of bytes from the stream, stopping at the end of the file.

virtual size_t readLine (char *buf, size_t maxCount, const String &delim="\n")
Get a single line from the stream.

virtual String getLine (bool trimAfter=true)
Returns a String containing the next line of data, optionally trimmed for whitespace.

virtual String getAsString (void)
Returns a String containing the entire stream.

virtual size_t skipLine (const String &delim="\n")
Skip a single line from the stream.

virtual void skip (long count)=0
Skip a defined number of bytes.

virtual void seek (size_t pos)=0
Repositions the read point to a specified byte.

virtual size_t tell (void) const =0
Returns the current byte offset from beginning.

virtual bool eof (void) const =0
Returns true if the stream has reached the end.

size_t size (void) const
Returns the total size of the data to be read from the stream, or 0 if this is indeterminate for this stream.

virtual void close (void)=0
Close the stream; this makes further operations invalid.


I'm guessing it probably won't work without writing some C++ code... any thoughts?

You certainly can use Datastream from within Python...

Perhaps steal from the following........

import ogre.renderer.OGRE as ogre
import ctypes
import random


class ds (ogre.DataStream):
def __init__ ( self, width, height, depth, format ):
ogre.DataStream.__init__(self)

## Note that this is for testing to ensure we get the right size buffer
self._size= ogre.PixelUtil.getMemorySize(width, height, depth, format )
self.buffer=[]
self.position = 0
for i in range ( self._size ):
self.buffer.append(random.randint ( 0, 255 ))

def tell ( self ):
print "**tell", self.position
return self.position

def seek ( self, pos ):
print "**seek", pos
if pos < self._size and pos >= 0:
self.position = pos

def size ( self ):
print "**size", self._size
return self._size

def skip ( self, pos ):
print "**skip", pos
temppos = self.position + pos
if temppos >=0 and temppos < self._size:
self.position = temppos

def eof ( self ):
print "**eof"
if self.position == self._size - 1:
return True
return False

def read ( self, dest, count ):
print "**read", count
if count <= 0 : return
pointer = ctypes.c_void_p ( dest )

for x in range ( count ):
pointer[x] = self.buffer[self.position]
if self.position == self._size-1:
break
self.position += 1



width = 240
height = 240
depth = 8
format = ogre.PixelFormat.PF_BYTE_BGR
requiredSize = ogre.PixelUtil.getMemorySize(width, height, depth, format )
mDs = ogre.MemoryDataStream ( requiredSize )
pointer = mDs.getPtr ()
for i in range ( requiredSize):
pointer[i] = random.randint ( 0, 255 )
img=ogre.Image()
img.loadRawData(mDs, 240, 240, 8, ogre.PixelFormat.PF_BYTE_BGR)

Andy

Thanks for the code, very helpful. I'm having a little trouble though. Here is what I have so far:


class FileObjectDataStream(ogre.DataStream):
def __init__ (self, file):
ogre.DataStream.__init__(self)
self._file = file
# calculate the size
self._file.seek(0, 2)
self._size = self._file.tell()
self._file.seek(0)

def read(self, dest, count):
print "**read", dest, count
pointer = ctypes.c_void_p(dest)
data = self._file.read(count)
for i in range(len(data)):
pointer[i] = data[i]
return len(data)

def skip(self, pos):
print "**skip", pos
self._file.seek(pos, 1)

def seek(self, pos):
print "**seek", pos
self._file.seek(pos)

def tell(self):
print "**tell"
return self._file.tell()

def eof(self):
print "**eof"
if self.tell() == self.size() - 1:
return True
return False

def size(self):
print "**size"
return self._size

def close(self):
print "**close"
self._file.close()


And I tried to open a mesh with the following code:


# Minimum required to create a mesh using the mesh manager
lm = ogre.LogManager()
l = lm.createLog("export.log", True, True, False)
bm = ogre.DefaultHardwareBufferManager()
rgm = ogre.ResourceGroupManager()
sm = ogre.SkeletonManager()
mm = ogre.MeshManager()

# Create a mesh
mesh = ogre.MeshManager.getSingleton().createManual("temp", ogre.ResourceGroupManager.DEFAULT_RESOURCE_GROUP_NAME)

# Open up a file
meshFile = open('box.mesh', 'rb')
# Feed it into the new DataStream class
meshDataStream = FileObjectDataStream(meshFile)
# Transfer the mesh from the hard disk to the mesh instance using the mesh serializer
ser = ogre.MeshSerializer()
ser.importMesh(meshDataStream, mesh)


So far so good, it gets up to the last line where it actually tried to import the mesh. but it trips up when trying to call the read function in FileObjectDataStream. Here is the output I get:


Creating resource group General
Creating resource group Internal
Creating resource group Autodetect
Registering ResourceManager for type Skeleton
Registering ResourceManager for type Mesh
**tell
**read <void * object at 0x00AC70C0> 2
Traceback (most recent call last):
File "test.py", line 67, in <module>
go()
File "test.py", line 65, in go
ser.importMesh(meshDataStream, mesh)
File "test.py", line 17, in read
pointer = ctypes.c_void_p(dest)
TypeError: cannot be converted to pointer


Not sure why it doesn't want to convert dest which was reported as type "<void * object at 0x00AC70C0>" into a pointer on the line "ctypes.c_void_p(dest)". It seems to me that the types do in fact match up. So I'm confused. If it doesn't work in my code then it shouldn't work in the code that you posted earlier either because this part is the same...

You read function probably needs to look something like (untested):
def read(self, dest, count):
print "**read", dest, count
pointer = (ctypes.c_uint8 * (count)).from_address( ogre.CastInt(dest) )
data = self._file.read(count)
for i in range(len(data)):
pointer[i] = data[i]
return len(data)

Andy

Thanks for your help again, but it's still not quite working. I've been banging my head against the wall for hours on this one... The problem with the code you suggested is that "ogre.CastInt(dest)" doesn't behave as expected. This is what I got (python code shown as >>>, output shown on next line):


>>> def read(self, dest, count):
>>> print dest
>>> print ogre.CastInt(dest)

<void * object at 0x009B90C0>
5


Which is clearly not the pointer I expected. So I changed that to:


>>> def read(self, dest, count):
>>> print dest
>>> add = str(dest)
>>> intAdd = int(add[-9:-1], 16)
>>> print intAdd

<void * object at 0x009B90C0>
10195136


Ok, looking better, but this is probably quite slow . Anyway, troubles are not over yet. The final code looks like this:


def read(self, dest, count):
print "**read", dest, count
add = str(dest)
intAdd = int(add[-9:-1], 16)
pointer = (ctypes.c_uint8 * (sizeRead)).from_address(intAdd)
data = self._file.read(count)
sizeRead = len(data)
for i in range(sizeRead):
pointer[i] = ord(data[i])
print pointer[i]
return sizeRead


And the output:


**tell
**read <void * object at 0x009B90C0> 2
0
16
**skip -2
Traceback (most recent call last):
File "importMesh.py", line 90, in <module>
go()
File "importMesh.py", line 87, in go
ser.importMesh(meshDataStream, mesh)
ogre.renderer.OGRE._ogre_exceptions_.OgreInvalidParametersException: OGRE EXCEPT
ION(2:): Can't find a header chunk to determine endianness in Serializer::determ
ineEndianness at ..\src\OgreSerializer.cpp (line 83)


I opened up the file it's trying to read with a hex editor and the first two bytes are indeed 0 and 16 so it seems to be reading them fine. The only problem could be that its not filling the dest buffer correctly. This is certainly more trouble than I was expecting...

Update: I tried a different approach to reading files using MemoryDataStream instead... But the problem is the same :/

Here is the code:


meshFile = open('box.mesh', 'rb')
data = meshFile.read()
dataSize = len(data)
storageclass = ctypes.c_uint8 * dataSize
bla = storageclass()
for i in range(dataSize):
bla[i] = ord(data[i])
meshDataStream = ogre.MemoryDataStream(ctypes.addressof(bla), dataSize)
ser = ogre.MeshSerializer()
ser.importMesh(meshDataStream, mesh)


I'm still getting the same ogre exception..


Traceback (most recent call last):
File "importMesh.py", line 100, in <module>
go()
File "importMesh.py", line 97, in go
ser.importMesh(meshDataStream, mesh)
ogre.renderer.OGRE._ogre_exceptions_.OgreInvalidParametersException: OGRE EXCEPT
ION(2:): Can't find a header chunk to determine endianness in Serializer::determ
ineEndianness at ..\src\OgreSerializer.cpp (line 83)