Python-ogre plus twisted

I am developing a mmorpg in python-ogre, and thus far have been through I think at least 5 near rewrites of the networking component. As an mmo, I feel that this is likely the most important part, so I want to get it right. I initialially used twisted with udp sockets, running in a separate process with a really bad ipc set up, when I started this project using the blender game engine. This system seemed inadequate and outdated for ogre, but I sort of ported it and reworked it when I made the jump to ogre. After this I went to basic sockets in udp, then sockets with tcp, and now I am pretty fed up with the socket api and all of the intricacies that can go wrong in any manner of wacky ways, so I am going back to twisted. Only this time, I think I can do a much better job of integrating it.

After several hours of trial and error, I finally managed to get python-ogre to play well with twisted. I was having a problem where my client would connect just fine to the server, but get no responses back from the server. It was one way communication.

My method of combining these tools, is to do all of my ogre initialization first, then initialize twisted. Finally, I use reactor.callLater to run my main game loop from the twisted main loop. Twisted gets to be king. My main loop handles rendering the screen. I have a server select window, so the twisted reactor does not connect right away, it connects after a server is chosen. This seems to work OK.

With the problem I was having, the same code would work if I ran it in text mode (without ogre), and fail after ogre initialized. I had a surprise when I tried running in opengl rather than direct3d mode - everything worked fine. I finally narrowed it down to one particular setting which seems to break twisted: Floating-point mode=Fastest. I changed this to consistent, and now my game is running well again (barring some events firing at random times and breaking some stuff, but this is application level and can be fixed).

I just wanted to post about this in case anyone else is having a hard time with twisted, to save someone the trouble. Also, there might be others out there who also use twisted, and I'm curious if there are any other curve balls I may not have hit yet. I haven't seen this mentioned anywhere (and boy did I look) so I assume I'm the first one to run into it.

I've been using PO + twisted in a commercial project for my employers ( sadly which has hit a dead end due to hardware sourcing woes ). I was using twisted to communicate between a PO display application and a flash control app that dictates what is seen on the display.

This is of course considerably smaller scale than an MMO, but I found the process in general to be quite simple to implement. Like you, I'm initalising everything Ogre before twisted, but I don't let twisted control the ogre rendering loop ( i use my own render loop which sits outside of both ogre and twisted, and controls both seperately ).

The flash element is really what controls everything, as the simple twisted server is set up to listen for changes sent from the controlling client, then update some flags for Ogre to catch and display in its next iteration. The advantage here is that I keep a very fast framerate in ogre, the downside being that updates are not instantaneous ( but are still only in the order of a second or so ).

All in all though, I'd say the PO / Twisted combination is very robust, and a good choice for any serious graphics and networking development.

I have some experience with this (I used python-ogre and twisted during my honours project), and have code samples I can post if you like.

One of the main issues with this is a problem twisted has with timings when using it in conjunction with the DirectX renderer.

In my experience the best idea as bharling said is to hand control of your applications main loop over to twisted and use a co-iterator to render frames.

If you want some specific code samples let me know and I'll dig them out for you, I was planning on eventually creating a tutorial about this for the wiki anyway.

I've been using Twisted + Python-Ogre as well for my project. I'd like to reiterate what Zyzle said. Use a Twisted Cooperator (twisted.internet.task.coiterate is the easiest way) to handle rendering one frame and doing user input stuff in Ogre. I've tried a few other ways of doing it and that one works the best by far.

I'm also writing an MMORPG-- or rather a more general virtual world framework that will support MMORPGs. My project is open source (MIT license) so I really suggest checking it out. I'd expect that basing your game on MV3D could save you a lot of development time.

By the way, I also had the same problem with Twisted + Direct3D. It's related to Direct3D mucking with floating point modes which changes how the Python time functions work. My fix has generally been to not use Direct3D, but I'm very happy to see you found a way to fix it while using Direct3D.

Anyway, I definitely suggest checking out MV3D: http://www.mv3d.com

Mike

I have checked out MV3d, it looks pretty nice but I am already so far along I can't do any large migrations at this point. If I had seen it a year or two ago I might have joined it instead of create my own Of course creating your own stuff from scratch is fun too, and I've learned a lot (and beaten my head against the wall a lot too).

I have seen the code for coiterate, I'm curious what advantage that method has to using callLater. I'll give it a shot, since it seems everyone feels it's the best method.

Also, bharling, I'm wondering how you manage to control twisted. I can't seem to tame the beast Is there actually a way to iterate a reactor from the outside? I looked for this for ages, with no luck. Although it also sounds like you are possibly using some kind of inter-process thing like I had with my first attempt.

I don't like opengl on windows, it generally makes everything look bad, at least on my computer. So I really wanted to be able to get direct3d to work!

Thanks everyone, I'll take a look at coiterate.

coiterate basically iterates the given iterator over and over until an allotted time limit (0.01s maybe?), and then gives control up to the reactor and the rest of the program until the next reactor iteration. I'm assuming you are doing reactor.callLater(0, renderOneFrame) or something similar at the end of renderOneFrame. That will only allow one frame to be rendered per iteration of the reactor. It's usually fine, but if you start doing a lot of things with the reactor (say receiving position updates on 40 objects), your framerate will decrease significantly.

Mike

i'm trying to learn twisted right now and perhaps integrating with PO later... twisted seems a bit daunting right now, so some code example would be awesome.

I suppose the simplest example would be something like this:

Basically you would initialize ogre in the usual way, however rather than calling startRendering you would create an iterator class like the one below:


class RenderIterator():
def __init__(self, root, window):]
"""The root is your applications Root object and window
the apps render window
"""
self.root = root
self.window = window

def __iter__(self):
"""If you dont know what this does read up on python
Iterators
"""
return self

def next(self):
"""On each iteration call the window messagePump and
render a single frame
"""
ogre.WindowEventUtilities().messagePump()
self.root.renderOneFrame()


Now in your 'main' method:


# make your game app stuff
# define/load resources etc
iter = RenderIterator(game.root, game.renderWindow) # create the iterator
coiterate(iter) # coiterate
# setup your twisted protocols/factories
reactor.run()


Twisted is a fairly complicated API but it does have some really great functionality, I would recommend reading into it more before trying to integrate it with a game as it can be very frustrating.

I will get around to making an article(on this and about a dozen other things :S ) in the wiki at some point however I've been neglecting bloop recently and really need to do some more work on it.

Thanks for the code. With callLater I have noticed that the framerate has very short but huge drops every so often, hopefully coiterate doesn't have that problem.

Actually, considering it only took me ONE day to integrate twisted and have my game running well again (and going from a 4 or 5 mob limit to a 400 mob limit before things start to lag), twisted's api really isn't that bad either. And the networking code is much shorter and clearer to boot. Twisted has a huge problem of documentation though, it's very confusing to know where to start with it. All those interfaces, various reactors which are under different namespaces etc. It fits it's name very well

I couldn't be happier with my networking performance now though. Now to fix all of the other problems

Twisted's core documentation is pretty good. Especially if you are looking to understand basic concepts. The Twisted Network Programming Essentials book is also good for a start. I especially like the explanation of Deferreds in there. However, some of the example code and such in the book is getting a bit out of date. A good way to understand how to use something specific in Twisted is also to just look at the code. It's generally well documented and usually easy to follow (though, with the reactor code, this isn't always the case as you noticed). Though that doesn't help if you don't know what class/function you need. In that case, there's always #twisted on freenode irc.

What's the theme of th MMORPG that you've been developing? Do you have a website I could check out?