ETL v3 development - Ogre Addon Forums

CABAListic

19-02-2008 15:15:21

So I've started some work on the development today. Not much as I'm also occupied by my own project, but anyway.

After some consideration I returned to my original intention to separate the data representation of the terrain from its visual representation. The Terrain class will be the data representation. It is a MovableObject, so it can be placed freely in your 3D world. You can query it for height values, do ray casts (which, if I can, I will improve in both speed and accuracy), and you can change its height array at will by setting individual values in the height array. The deformation via brushes will be realised via external methods which make use of that.

From the Terrain class you can then create a visual representation which will itself be a MovableObject, but you can "attach" it to the Terrain so that it will take the same position and orientation as the Terrain class. I even plan to allow to give ownership of the visual representation to the Terrain class so that all you need to do is a single line for the creation of the visual.

The Terrain class exposes a set of functions which directly describe the vertices of the terrain. There's a getNumVertices() function over which you can iterate and then retrieve the vertex's position, normal etc. via according functions. ETL will use this itself for the visual representations, but you can also use it to create physics shapes of the terrain (should defintely simplify the process a lot). Subparts of the vertex description are available via a wrapper class.

CABAListic

20-02-2008 17:26:03

I just finished the Terrain class in manners of functionality which I think it needs to provide. Might be that I need to add or change some things later. Anyway, the class compiles, but is totally untested yet. I have no idea if my code to retrieve terrain information in world space is in any way correct

In local terrain space, there's not that much changed. The code to get the terrain height at a given position is now accurate to the triangle instead of an interpolation as it was before. I figured that the accurate version is actually shorter in code and needs a similar amount of calculations as the interpolation.

If anyone wants to check it out, you can get it from my bazaar repository (http://bazaar-vcs.org):
bzr checkout http://repository.oddbeat.de/ogre/etl
But at this moment that would really be mostly for curiosity. There is nothing you can use outright, the build files are out of date (except the Code::Blocks one I'm using to develop).

CABAListic

21-02-2008 20:24:06

I'm nearly finished with the Patch class as the terrain rendering primitive, so I'm slowly nearing a stage where I can actually see and test the terrain again

In the process of rewriting the Patch class, I also spotted a bug in the old code. I always had the feeling that LOD morphing with ETM didn't work quite as well as with the TSM, and I was correct - the morph factor kicks in too late and is too small due to a scaling error. D'oh.

CABAListic

22-02-2008 15:37:54

So, much of the terrain functionality is done, and in theory everything would be ready to create and display some landscape. Unfortunately, that's in theory only as I've hit one of the dreaded heap corruption bugs. I'm receiving SIGSEGV and SIGABRT, and the debugger is no help at all. Damn it, I hate those!

CABAListic

22-02-2008 17:46:43

Found the error, it was a double deletion in my Array2D class due to a missing copy constructor. Actually, I had provided one, but a templated version, so that I could copy from other arrays as well. Apparently, the compiler is not as clever as I had thought. After providing a non-templated copy constructor, at least this error went away. Still some debugging to do though

CABAListic

22-02-2008 20:36:20

Hah, finally got it working:

Btw., patches no longer need to be square, either. You can construct a rectangular terrain with rectangular patches, if you wish. Don't know if it has much practical use, but then again, you never know.

The terrain in the picture was created like this:



      ET::Array2D<float> heights (129, 129);

      for (int i = 0; i < 129; ++i)

      {

        for (int j = 0; j < 129; ++j)

        {

          float x = float(i)/16;

          float z = float(j)/16;

          heights[i][j] = Math::Sin(x) * Math::Cos(z);

        }

      }

      mTerrain = new ET::Terrain("MyET", heights, Vector3(10, 60, 10));

      SceneNode* terrainNode = mSceneMgr->getRootSceneNode()->createChildSceneNode(Vector3(0,0,0));

      terrainNode->attachObject(mTerrain);

      ET::Page* page = new ET::Page("MyETPage", mTerrain, 17, 65, true);

Terrain is the pure description of the terrain, it does not render anything at all. But as you see, it can be aded to the scene node. Now if you actually want to see something, you need to get Patches on the screen. Simplest way to do that is the one shown, just create a Page. The Page takes a name (base name for the patches), a pointer to a TerrainDescription object and the patch sizes in X and Z direction. The last parameter specifies whether the Page should be automatically deleted when the TerrainDescription object is destroyed - i. e. you don't need to care at all for the Page! The Patches of the Page will automatically be added to the same SceneNode as the Terrain object, so in fact you really only have to deal with the Terrain.

Code is in the bzr repository if you want to try it out. Next thing to do for me is to complete the Page class, add ray scene queries back in and add the Brush interface.

kungfoomasta

22-02-2008 20:54:02

Not sure if we're allowed to post on here..

Good job! Its awesome to see attaching to scene node works.

Some questions:
- What is a Page?
- Why does a Page have to be created before we see anything?
- Where does the material come in?

CABAListic

22-02-2008 21:33:46

Not sure if we're allowed to post on here..
Go ahead, if I hadn't wanted comments I'd have locked the thread

- What is a Page?
It's a collection of Patches, basically it's the visual part of the old TerrainManager. It takes a TerrainDescription as argument, divides it into equally sized subparts and creates Patches from these subparts. (The Patches are the rendering primitives, i. e. that which was previously called Tile).

- Why does a Page have to be created before we see anything?
Because I wanted to separate the data representation from its visualisation. It allows for greater flexibility which might come in handy for writing a paging solution on top of it or so. In theory, you could even write a SphericalTerrain derived from TerrainDescription which models something like planetary terrain. Then you could still use Pages and Patches with the new SphericalTerrain without any modification.

- Where does the material come in?
Page will have a setMaterial function, alternatively you will also be able to access the Patches of the Page and assign materials individually.

kungfoomasta

22-02-2008 21:44:13

Thanks, that makes sense. Will any issues arise if you create the Page before adding the Terrain to the SceneNode?



mTerrain = new ET::Terrain("MyET", heights, Vector3(10, 60, 10)); 

ET::Page* page = new ET::Page("MyETPage", mTerrain, 17, 65, true); 

SceneNode* terrainNode = mSceneMgr->getRootSceneNode()->createChildSceneNode(Vector3(0,0,0)); 

terrainNode->attachObject(mTerrain);

How flexible is the process for defining patch sizes? Is there a reason you picked 17/65, or can we make up anything we want? (40 x 40, 50 x 50, etc.?)

Will the patches overlap, or does that last row/column of patches get truncated?

Page will have a setMaterial function, alternatively you will also be able to access the Patches of the Page and assign materials individually.

This means we can use a different set of splatting textures per patch and not get penalized? (for example, 2 passes for each patch, but each patch could have 8 different splatting textures?)

CABAListic

22-02-2008 22:35:31

Thanks, that makes sense. Will any issues arise if you create the Page before adding the Terrain to the SceneNode?
No. The Patches register to the TerrainDescription (and Terrain is a TerrainDescription, i. e. derived from it) as a listener. The listener interface informs them whenever the attachment of the Terrain object changes, and they reflect this change. Visibility is similarly adopted. So you can attach the Terrain object to whatever you want, whenever you want, the Patches will reflect the change.

How flexible is the process for defining patch sizes? Is there a reason you picked 17/65, or can we make up anything we want? (40 x 40, 50 x 50, etc.?)
It's still the old requirement that the sizes satisfy the formula (2^n)+1 for any positive n. This is necessary for the LOD mechanism to work - every further level skips every second vertex from the previous level which gives you this formula. n is incidentally the maximal number of different levels possible.

Will the patches overlap, or does that last row/column of patches get truncated?
They need to overlap to form a seamless terrain, so that's what the Page class does. You can manually create some and do otherwise, but then you'll have a gap in the terrain.

This means we can use a different set of splatting textures per patch and not get penalized? (for example, 2 passes for each patch, but each patch could have 8 different splatting textures?)
Yep. In fact, I plan to write a higher level splatting manager which will exploit this to optimise the splatting for speed

CABAListic

23-02-2008 02:07:40

I committed a draft of the ray scene queries. I'm using a different approach now than before. The previous approach was taken from Ogre's TerrainSceneManager where the code just moved along the ray in small steps and tested when it reached a point below the terrain. This is somewhat cheated, furthermore for very large terrain scales (i. e. large terrain extents in Ogre units) the small step would take quite a long time under certain conditions.
Instead I now follow quads of the terrain along the ray with the help of Bresenham's line algorithm, and for each quad touched I test for triangle level collision. With this method the collision detection becomes accurate to the triangle level and also works for collision detection from below (although that's usually quite pointless). Speed seems to be on par with the previous approach, though I'd expect better behaviour in the aforementioned bordercases.

There's still a bug, though. The ray query only works correctly with the terrain at the origin, i. e. not moved or rotated in any way. If the terrain was moved, the rays work only partly, but often incorrectly return false. I assume that the other world space queries don't work, either, but haven't tested them yet. I'll investigate tomorrow

CABAListic

23-02-2008 18:19:14

Finally got the ray scene queries to work correctly (or so it seems), I had to modify my approach slightly. Anyway, the Ogre head of the demo is now again following the terrain at the mouse cursor, even if the terrain is rotated and moved

I'm off to implement the Brushes now.

CABAListic

24-02-2008 02:21:47

So I have a somewhat working Brush class, and I added deformation support to the Terrain. However, the results are a bit disappointing, deformation performance is way behind the level I achieved with ETM v2. It seems that the indirection via listeners, subarea classes and virtual function calls really hurts performance here. I'll have to think about how I can make it any faster.
The listener interface does have the advantage that I can limit the update rate of the visual representation, so it only updates 30 times per second or so. That way I can achieve the previous performance, but that's still not good.

CABAListic

24-02-2008 02:45:36

Never mind. I accidentally forgot to skip out of the update if there was nothing to do. In result for every deformation step ALL patches were updated, no matter if they were touched by the deformation or not.
With that error resolved, performance is back where it was, plus with the deferred update the performance drop becomes quite neglectibe

kungfoomasta

24-02-2008 02:53:28

Awesome! You're really moving fast with this.

Are you going to add in the smoothing functionality posted in the forums, or will that be a user added feature?

How does the interface look? Do you create a Brush class, and use an instance of the Brush to deform the terrain?

CABAListic

24-02-2008 03:15:21

Pretty much, yes. Creating a Brush goes like this:



ET::Array2D<float> array (16, 16, 1);

ET::Brush* brush = new ET::Brush(array, Vector2(10, 10));

Just like the terrain it is created from a 2D array which you can fill however you like (there will be utility functions to load from image, raw etc., just like before). The Brush has its own scale (2nd param) and can be rotated around its own normal (not in 3D space, though, that would not work).

Deforming terrain could then look like this:



std::pair<bool, Vector3> result = mTerrain->rayIntersects(ray);

if (result.first)

{

  float brushIntensity = evt.timeSinceLastFrame * 5.0;

  terrain->deform(result.second, brush, brushIntensity);

}

As you can see, you pass in the 3D world space position directly, no need to transform to a vertex index. And you can actually do "fluent" deforms, i. e. the arrays of Brush and Terrain don't need to overlap, values are interpolated.

CABAListic

26-02-2008 23:49:32

Been a while since the last update, and there's currently not much progress. I am at a crucial point with my own project, so this is where my attention currently goes. Since ETM v2 works quite well, I hope a delay is not too troublesome

Anyway, I did start on implementing what I call a DynamicTexture. It's a generalisation of what the SplattingManager does and will therefore build the foundations of the new SplattingManager. The DynamicTexture is really just a wrapper around Ogre's texture class which additionally keeps a 2D array of ColourValues. You can use Brushes to change the colour at any place of the array, and the change will then be reflected to the actual texture. The SplattingManager will build on top of this functionality, but you can also use this class directly to create a ColourMap for your terrain.

CABAListic

27-02-2008 07:19:05

Beautiful, isn't it? There are a couple of reasons that make this new DynamicTexture superiour to the previous workings of the SplattingManager. For one, you can freely specify the pixel format, so you are not restricted to byte-sized textures, you could also use short or float textures. The DynamicTexture doesn't care, it lets Ogre handle writing ColourValues in the appropriate pixel format. The SplattingManager later will just determine the number of channels from the given pixel format and not care beyond that, either.
In addition, the DynamicTexture remembers its dirty region, so only the part of the texture that you have edited will be written to the texture buffer, greatly increasing its speed. DynamicTexture is a lot more friendly on your framerate than the SplattingManager ever was, and that's even without deferred updating which should make the framerate hit nigh unnoticable

CABAListic

02-03-2008 20:09:56

Splatting is back. This screenshot shows a combination of splatting and a colour map for additional slight colour modifications:

The splatting is done with the help of the new class SplattingLayout. This works pretty much like the previous SplattingManager, but it's only the foundation for the new advanced SplattingManager which will be a lot more clever than before:

Implicit number of channels: You will no longer need to specify how many channels you need. When splatting, instead of a channel you pass in a texture name, and the manager will automatically reserve a new channel for it if it doesn't know that texture yet.[/*:m]
Ability to divide the map: This feature allows to divide the splatting map into several different parts. Each part will manage its own mapping of texture names to channel numbers, so this can be used to create optimised splatting layouts for your patches.[/*:m]
Freeing of unused channels: Blank channels are automatically tagged as such and reused when you splat with a new texture. So if you had used one texture before, but due to other splattings its channel is completely blank, that channel will be marked unused. There will also be the option to reorganise the channels at the end, so that any remaining free channel will be filled with the ones following after so that you have no gaps in your splatting maps.[/*:m][/list:u]

Dibalo

02-03-2008 23:03:39

Sounds very promising!

Does this "flexible" splatting affect the performance? I mean with fixed splatting you can use 4 splatting textures in only 1 pass. Is this possible with this new "flexible" splatting? Does it add new passes only when itÂ´s necessary?

CABAListic

02-03-2008 23:15:17

Yes, it will affect performance - it will most definitely improve it because textures are only splatted on patches where they are actually present. This reduces number of required passes and number of instructions. Of course it requires a slightly more sophisticated material setup. ETL will provide a small "library" of splatting shaders for all eventualities.

Alternatively, you can still use it like before, don't worry.

Laser

02-03-2008 23:16:48

Great stuff

kungfoomasta

04-03-2008 19:29:34

This new release is going to be incredible!

Are you waiting for the editor to make a release? If possible, I'd like to get it before then, I can integrate it and provide valuable feedback that might be beneficial to add in before you officially release it. I am not using any splatting at the moment, so I would be mainly involved with creation of the terrain, and accessing the terrain data to create my PhysX collision object.

syedhs

04-03-2008 19:40:10

Yeah incredible.

I will definitely going to take a peek at this library and of course, use it.

CABAListic

04-03-2008 19:48:22

@kungfoomasta: You can always take a peek at the repository, I'm not holding any code back on my local PC

Just look somewhere at the beginning...
Terrain code is, in principle, immediately usable, but it's still lacking the utility functions to actually load heightmaps, so for the moment you'd have to manually provide the contents of the 2D array. It's also currently lacking a Visual C++ project file, but this shouldn't be much of a problem - just create a static library build, add all the files in src/ and include/ and provide the Ogre include directory.
I'm not moving as quickly as I'd want with this library, but I really need to get along with my actual game project, so ETL has to take 2nd place for now. Since ETM v2 is quite usable, I think you will manage. But in the end, ETL is hopefully going to rock!

kungfoomasta

04-03-2008 20:02:45

I forgot you're using Linux.

I think I'll grab it after the heightmap to float array interface has been defined, I can wait. Part of my response was seeing the status of ETM, so thanks for updating us!

CABAListic

07-03-2008 11:20:19

I've added utility functions to load heightmaps from images and raw data, although I've not yet had the chance to test them. So aside from possible bugs yet to squish, they should be a lot more robust. Internally, loading from raw data and from image share the same piece of code. Also, this code now takes endianess of the system into account (not that I know of any big endian system relevant to 3D graphics programming, but still).
The saving counterparts are still missing, but anyway, progress is progress

kungfoomasta

07-03-2008 17:43:56

Awesome, I'll Look into this over the weekend. Thanks for adding this in so quickly!

Espire <EmasteR>

09-03-2008 21:29:16

CABAListic

10-03-2008 15:40:49

hello i just want to ask, it is hard to save terrain made with all that functions into 1 file and than load with ogre ? if yes than it is hard or not ? will i learn it after working with ogre ? i mean monts not days :p
You need several separate files to describe a terrain fully - one heightmap, several splatting maps and probably some sort of configuration / description file. ETL will be able to load and save heightmap and splatting maps, but the rest is up to you. If you want to get it all into a single file, you need to create some sort of archive. Neither Ogre nor ETL will help you with that.

Espire <EmasteR>

11-03-2008 12:03:58

willthrom

12-03-2008 08:44:29

Hi, first I want to apologized for my english

I'm looking for a SceneManager for a Flight Simulator. I tested PLM2, but its Doc is very poor and like as a abandoned proyect.

Now, I try to use ETM. I'm not interesting in Edit terrain at first place (only if a boom is dropped and crater is created).

In a Flight Simulator we need large landscape. I try to create a map 45000 x 45000 Ogre Unit with 2048x2048 vertex but my STUPID card is very low when a look up the horizont.

Now, I have HGT from all the wolrd. This files are georeference hightmaps (latitude, longitude). I can convert this file to PNG Grey scale or create a plugin to allow read it in runtime, but it's nos the big problem, I need to know the ETM capabilities. Here are my questions:

When I look the horizont change the vertex num which are very far? or always ETM render 2048+2048 vertex?
When I move arrond the world, sometime the camera go through terrain a sometimes the camera go up (stamp to terrain). What is it the problem?

I have to create a Terrain Manager to load the next heighmaps in runtime when I move to the next maps, but I don't understand well whe Page system in ETM (is it like PLM2? divide the terrain in run time in small pieces? or not?.

Thank you very much

CABAListic

12-03-2008 08:52:52

Put quite simply, ETM is not suited for your needs. For large terrains you absolutely need a paging solution, and ETM does not and will not offer paging anytime soon. The new upcoming ETL provides the low-level capabilities to build a paging solution on top of it, but you would have to do that all by yourself.

Apart from that, yes, ETM does reduce the number of vertices rendered for far-away pieces of the terrain (via level of detail). And if by the camera going through terrain you refer to the ETM demo, then yeah, there's some kind of bug I can't for the life of me figure out. It's not the fault of the terrain height function far as I can tell.

willthrom

12-03-2008 10:38:52

if new ETM support low-level page I think i can spend time in develop a solution with GeoTiff/Terragen/STRM Support, i have some experience.

I don't know if the low fps issue when i set 2048x2048 vertex is for my Ati x1550 PCI_E x1, but i worried about vertex performace.

when New Release will be out?

CABAListic

12-03-2008 12:28:11

Not anytime soon. You can follow this thread for progress (which is its actual purpose).

Anyway, 2048x2048 vertices are simply too much for a non-paging solution. For one thing, they will be kept in memory, and even with level of detail there's just too many patches rendered for good framerates, especially if you're using a splatting technique. Only if you were looking top-down on the terrain, then patch culling might result in acceptable framerates.

Just to clarify, low-level primitives for paging just means that the new version supports multiple terrains next to each other with proper LOD transitions. You would be fully responsible for loading terrain pages at the correct time, unloading them, placing them, managing your entities etc. pp.. And if you want to do advanced stuff like horizon culling (which may or may not be beneficial to a flight simulator), then you need to write your own scene manager based on ETL.

CABAListic

23-03-2008 23:12:42

Sorry for the lack of updates, been a bit busy (and lazy, at the same time). I just committed utility functions for saving real arrays. So now you can both load and save heightmaps as well as brushes. I know it's not spectacular, but anyway

CABAListic

24-03-2008 15:53:08

And another small update: colour arrays can now also be saved and loaded. This allows to store and restore splatting maps, colour maps etc.. Functionally ETL is now almost where ETM is, plus it's already more flexible

I need to finalise some minor bits of the current code and add lightmap and base map generation, then ETM v2 functionality is complete.

CABAListic

30-03-2008 22:32:38

I implemented a function to generate a normal map for the terrain; you'll need one if you're going for dynamic lighting since using the actual vertex normal will clash with terrain LOD. It's working, but I'm not entirely satisfied with the results yet.

Anyway, next thing I'll get to is probably a sort of dynamic splatting material implementation. The way I'm currently planning to do this has you give a material to the SplattingMaterial class, and there it will make a copy of this material and add splatting passes to the best available technique. This approach makes the code a lot more generically useful compared to creating the whole terrain in the class. This way you can define arbitrary passes after the splatting passes and deal with shadows etc. however you need.

MOROHEUZ

15-04-2008 11:52:02

Great stuff .. CABAListic

Can you post a Status @ moment ?? % or whatever :-)

The 2.2 is great @ moment too ...
but the 3.x has more fps i think ??

greetz

Moro

CABAListic

15-04-2008 13:13:52

No, not really, ETL v3 will have pretty much equal performance. It can sustain higher frame rates when modifications (terrain deformation etc.) are taking place, but this is due to a trick of carrying them out at a slower rate than the frame rate.

Anyway, progress is a bit slow currently. I've got the normal map right, finally, and generating tangent and binormal maps will be easy to add. I'll then carry over the shadow map generation code, at which point ETL v3 has all the features of ETM v2. So that's probably a good point to include a decent build process and maybe release an alpha version. Anyway, after that there's still enough to do, at the top a splatting material handler and the more intelligent splatting manager. But these are not necessarily required to use ETL v3, so you'll have to weigh the percentage for yourself

kungfoomasta

15-04-2008 18:23:24

Anyway, progress is a bit slow currently.

I know how you feel! I wanted to test out the latest ETM but my hands are tied with QuickGUI at the moment.

Keep us posted on any developments; I like reading about ETM development, including the editor and SongOfTheWeave's lighting implementation.

Yacoby

03-05-2008 14:09:03

How complete is this? I did try compiling it but didn't manage to (63 errors

)

In other words, is it worth developing with this verson, or just say with 2.x for now?

CABAListic

03-05-2008 15:04:02

If you can't get it to compile outright, I would advise you to stick with v2. There's no immediate advantage to using v3 over v2 just yet to surpass any compiling trouble, and a final release of ETL v3 is still a while off, I'm afraid.

tenttu

06-05-2008 15:44:01

I made a brief look into whats the state of ETL v3 and would have some suggestions.

FrameListener should not be used within components like these. It is very likely that a project does time management on it's own.

MovableObject should not be inherited from, when you only need a position of the object. Let the user to deal with the conversion and take local position as input. There should only be one MovableObject in ETL.

Creating Array2D container from scratch is not a very good idea. It should either be extended from stl container or vector<vector<>> should be used.

There is no point in creating binormals with CPU, they will just use memory. It's very fast to calculate them on GPU from tanget and normal. It's also possible to approximate tangets at GPU for convex surface like this.

Brush should be just an interface, which could be used to create different kind of brushes. Such as image based or mathematical ones (circles etc)

Doing AABB culling per patch level, especially when patch size is 32x32, is more expensive than rendering them all. I would put the culling on Page level.

Using the current CPU heavy lodding is the worst rendering performance bottleneck in ETL. Modern GPU can render the terrain faster without any lodding. It should be possible to easily turn of the lodding and creation of lod buffers as lod meshes also eat up alot of memory for pretty much nothing.

Tri-strip terrain without lods would allow the best performance and minimise memory usage. However editing in this mode would be hard (slow).

Is there a reason why you are using bazaar instead of SVN? As it would be a lot easier to submit patches with it. ( I have no idea how to create patch against bazaar repository

)

kungfoomasta

06-05-2008 18:01:22

MovableObject should not be inherited from, when you only need a position of the object. Let the user to deal with the conversion and take local position as input. There should only be one MovableObject in ETL.

Why is this? In the default TSM each Tile is a MovableObject, no? I assume this because with raycasting you can select world pieces and see their bounding boxes. If MovableObject is not inheritted from, standard RayCasts will not work. In addition, everything that has a position and is rendered in the scene is a MovableObject according to Ogre's design, why not make things consistent? Please give us more information why you feel this way, I was one of the people pushing to make use of the Ogre MovableObject class.

CABAListic

06-05-2008 18:13:14

Thanks for your comments.

FrameListener should not be used within components like these. It is very likely that a project does time management on it's own.
Doesn't matter, as in almost all cases the FrameListener will still do the job it's supposed to do (which is deferred updating of vertex and pixel buffers), even on custom render loops (unless you forgo Ogre's calling of the framelistener somehow or do not render a frame regularly). Even so, you can just tell the object on construction not to do automatic updates, and then it won't register as a frame listener. You'll have to call update() on the object manually, though.

MovableObject should not be inherited from, when you only need a position of the object. Let the user to deal with the conversion and take local position as input. There should only be one MovableObject in ETL.
It's not just position, it's also orientation etc.. The point is to be able to attach the editable objects to SceneNodes, as that's Ogre's way of organising scene content, and the editable objects should be synchronised with displayed content to spare the user of having to translate editing coordinates. You don't actually need to attach it to a Node so long as you stick to local coordinate space.
In any case, this functionality was requested and I'm not sure where's the harm in it.

Creating Array2D container from scratch is not a very good idea. It should either be extended from stl container or vector<vector<>> should be used.
You can't extend STL containers, they are not made for inheriting. I would have used boost if Ogre had adopted it as a dependency, but otherwise a vector of vectors is a workaround and clumsy to use, imho. ETL needs too many 2D arrays as to make that a comfortable solution. Or in short: STL does not have a suitable container for multidimensional arrays, and therefore I do think it justified to write one.
Even so, there's likely some things you can improve on it, for one thing its creation is not yet as efficient as it could be. And conversions to and from a vector of vectors would be a small thing to do.

There is no point in creating binormals with CPU, they will just use memory. It's very fast to calculate them on GPU from tanget and normal. It's also possible to approximate tangets at GPU for convex surface like this.
Then don't. Binormals are not calculated unless you request it. And the few if's on editing updates won't hurt your framerate, promise

I just included them for completeness's sake, for all I'm concerned, any vertex normal/tangent/binormal is quite useless unless you disable LOD.

Brush should be just an interface, which could be used to create different kind of brushes. Such as image based or mathematical ones (circles etc)
I will think on it. Right now, my usage of the brush forbids it to be an interface, but I think it can be rewritten to that notation. It would most definitely be cleaner, no doubt.

Doing AABB culling per patch level, especially when patch size is 32x32, is more expensive than rendering them all. I would put the culling on Page level.
ETL doesn't cull anything, it lets the SceneManager do with that. The very same way that ETM v2 and Ogre's TerrainSceneManager deal with it, ETL just gives the patches as MovableObjects and Renderables which are usually culled by the OctreeSceneManager. And I very much doubt what you're saying. A patch is usually one of the more expensive objects to render in the scene due to splatting, and therefore any patch not rendered will be good for your framerate. If Octree culling were so expensive, it would be quite useless.

Using the current CPU heavy lodding is the worst rendering performance bottleneck in ETL. Modern GPU can render the terrain faster without any lodding. It should be possible to easily turn of the lodding and creation of lod buffers as lod meshes also eat up alot of memory for pretty much nothing.
Beg your pardons, but ETL uses GeoMipmapping, and that one's been designed specifically to handle LOD on the GPU. Might be you don't fully understand the approach: Each patch has a fixed vertex buffer which is never modified unless via terrain edits. Whenever Ogre requests a render operation for a patch, the camera squared distance to the center of the tile is calculated (this is negligible), and based on that an according LOD level is chosen. The level of detail simply determines which index buffer goes along with the vertex buffer, and since the vertex buffers' layouts are all the same, they share the index buffers. If an index buffer for a requested LOD does not yet exist, it is created, but only once.
And that's really all there is to it, all that happens is a switch of index buffers at LOD transitions. Any other necessary calculation is only done when the terrain is edited.
Still, setting max LOD to 0 should effectively disable any effects of LOD entirely (but it WILL hurt your performance badly unless your camera only shows close terrain). There'll be only one index buffer generated that way.

Tri-strip terrain without lods would allow the best performance and minimise memory usage. However editing in this mode would be hard (slow).
The TerrainSceneManager has tri-strips. But they are immensely more complex to generate correctly, and sinbad has stated time and again in the forums that on modern graphics cards, there is no measurable performance gain from them, sometimes even the opposite. Therefore I won't implement it.
It's nothing to do with editing, though. The triangles purely affect the index buffers, while any terrain editing only affects the vertex buffers and doesn't care for the way triangles are generated.

Is there a reason why you are using bazaar instead of SVN? As it would be a lot easier to submit patches with it. ( I have no idea how to create patch against bazaar repository

)
I like bzr more, simple as that

It's easier to use (except for lack of tools, maybe), it works locally and remotely without server. 'bzr diff' should output any changes you made to the repository in usual diff format.

tenttu

07-05-2008 11:18:35

#include <vector>



template <typename T>

class Array2D

{

public:

  Array2D(int rows, int cols)

  {

    for(int i=0; i<rows; ++i)

    {

      mData.push_back(std::vector<T>(cols));

    }

  }

  

  // other ctors ....



  inline std::vector<T> & operator[](int i) { return mData[i]; }



  inline const std::vector<T> & operator[] (int i) const { return mData[i]; }



  // other accessors, like at() ...



  void resize(int rows, int cols)

  {

    mData.resize(rows);

    for(int i = 0; i < rows; ++i)

      mData[i].resize(cols);

  }



  // other member functions, like reserve()....



private:

  std::vector<std::vector<T> > mData;  

};

In general use tri strips can be slower if the material changes causes the buffers to be split into multiple parts and if the model is hard to split into tristrips. But terrain can easily be divided into tri strip and only has one or few materials. The performance gain from tri strip rendering is neglible, but the memory saving is considerable when the terrain size is big.

I do understand how geomipmapping works and have implemented in numerous times, back in the year 2000 when we were using Voodoo2s and such. Current generation GPUs (and all that support ps2.0a/b->) are just hungry for triangles. If you throw in static buffers of triangles it's hard to reach any kind of limit on the GPU side. Recent games that doesnt use any kind of lodding for terrain are Quake Wars, UT3 and many more.

Of course the culling can be done multiple ways and I just used AABB as the worst case. Octree obviously speeds things up, but it still won't be the ideal way to handle over 1000 objects. Of course you could increase the Patch size to get better results, but it soon reaches the ideal size of a Page.

CABAListic

07-05-2008 12:20:11

You are correct, I forgot that renderOneFrame obviously also calls the FrameListener functions. However I still think this is a overkill for the purpose. It creates unecerary function calls per frame when no terrain editing is done. Update should be done by user and delayed if he feels thats necesary. User also has better knowledge wether the terrain needs updating or not.
As I said, just disable automatic updating, and the functions will never be called at all. It's simply a convenience method to ease the use of ETL for most people. If you don't need it, disable it and you won't pay for it. Besides, in my demo there is absolutely no performance difference between enabled or disabled auto-updating for terrain display, however, once you begin to edit terrain, the automatic deferred updating yields framerates which are far superiour.

Of course the best way to implement terrain is by using MovableObject. However it's not currently used the way it's suppose to. For example inheriting DynamicTexture and SplattingLayout from MovableObject, just to retrieve the correct world position is not correct. Inheriting is a powerful tool but should not be used without care.
Not correct in what way? Both are designed to be placed on top of the terrain, so usually they'll share the terrain's position and orientation and are therefore movable. MovableObject provides that notion and allows to gather Terrain and SplattingLayout on the same Node so as to keep their position and orientation synchronised. In any other case I would have to either duplicate that functionality (and the user would either have to manually sync the positions or the Layout would have to depend on the Terrain class) or only provide local 2D coordinate editing and have the user figure out how to get from 3D coordinates to local 2D space (like it's with ETM).
Even so, you can ignore the MovableObject part if you only use the local 2D editing functionality, you don't need to attach it to anything, therefore, no MovableObject functions will be called at any time. MovableObject does bring a few variables of its own, but you'll be hard pressed to create enough DynamicTextures to notice.

If Patch is inherited from MovableObject, on a 1024x1024 terrain with 32x32 patch size it will cause 1024 MovableObjects to be created. Raycasting and AABB culling would go through all of these objects and that would slow down the rendering alot.
Then use a larger patch size. The only reason to use smaller patch sizes is for culling and more effective LOD. Since you seem to want to use neither, set your patch size to your full terrain's size, and you'll even save some memory at the places where the vertex buffers would otherwise overlap. But I'll guarantee you, you will not be happy about your framerates. You *will* notice whether your graphics card renders only half of the 2 million triangles or all.

The way I would implement the structure is to inherit Page from MovableObject and Patch from Renderable. Nothing else would be MovableObject or Renderable. This system could be compared to Entity/SubEntity for example.
As said, that makes no sense, because you can just choose the Patch size accordingly.

You can extend STL containers, however the extending is not done by inherting, but by implementing certain interface. Though my statement was a bit brief and what I meant was to make Array2D to use vector instead of raw arrays. Below is a simple example how it could be done.
vector represents a dynamic array. My Array2D container does not need a dynamically sized container, so what's the point? The reason to use vector over raw arrays is not to have to deal with memory management, but Array2D encapsulates that just as well. Using a raw array has the advantage that 1) the memory is exactly as large as it needs to be while vectors will reserve some additional size, and you seem to be fond of memory savings, and 2) the memory is one continuous block while in the case of a vector of vectors you get several different.

In general use tri strips can be slower if the material changes causes the buffers to be split into multiple parts and if the model is hard to split into tristrips. But terrain can easily be divided into tri strip and only has one or few materials. The performance gain from tri strip rendering is neglible, but the memory saving is considerable when the terrain size is big.
The terrain size is irrelevant, I already stated that the index buffers are shared among the patches. So the patch size would be relevant here. Let's say we have a 65x65 patch, then we'd have 4096 triangles. That would require 12288 indices for trilists and 4098 for tristrips. Since indices are 16bit integers, a tristrip would save you 16 kbytes, which may add up to about 160 kbytes if you're using LOD (which you don't seem to). Well, it's a save, but it's not really spectacular, imho.
Of course, with larger patch sizes the saving would increase steadily to noticable values, but since I do believe in culling and LOD, those patch sizes are not recommended either way. You're free to change the index buffers to use tristrips, I'll happily test such a change and include it if it does not screw anything up.

I do understand how geomipmapping works and have implemented in numerous times, back in the year 2000 when we were using Voodoo2s and such. Current generation GPUs (and all that support ps2.0a/b->) are just hungry for triangles. If you throw in static buffers of triangles it's hard to reach any kind of limit on the GPU side. Recent games that doesnt use any kind of lodding for terrain are Quake Wars, UT3 and many more.
As stated above, just do. Set the patch size to your full terrain size, set max LOD to 0, and you'll pay neither for culling nor for LOD. But I guarantee you that you'll be surprised to find that you're wrong about your performance believes.

khawkins

11-05-2008 19:15:14

Is there any particular reason to not just derive from Ogre::Entity? instead of Ogre::MovableObject?

I started using ETM for my project, but I just hit a snag with raycasts that I'll have to provide a workaround for because ETM's Tile derives from MovableObject. I need access to the polygon level for intersection tests, and I wouldn't need to build a workaround if Tile derived from Entity.

Not a huge deal (other than adding an extra 8 or so hours to my dev time), but I'm curious about the design decision.

CABAListic

11-05-2008 19:36:54

khawkins

11-05-2008 20:10:49

Because Entity is a complete type not meant for derivation. It has no virtual destructor and no hooks that could be usefully overridden. Even if it could, it offers way too much functionality for what is needed for a piece of terrain (like skeletal animation, subentities etc.).

Thanks. I could see an occasional use for those other things like skeletons and subentities on terrains, but that gets more into full-fledged world definition than the terrain system with ETM, so it's a good point.

In any case, ETM provides its own raycasting functionality. And while it's an approximation in ETM v2, it is exact to the polygon level in ETL v3.

I can live with the approximation implementation, but polygon level is a desired feature. Any estimates for v3 release?

Also, thanks for the cool library. It's been helpful.

Kali

28-05-2008 23:31:21

I wanted to give this a try but had to get it to work on linux first. So I decided to just do it right and created a patch for a proper Makefile project.

Some small remarks though:
- I didn't get ETTerrainPatch.cpp to compile, but seeing as the Sample doesn't require it, I assume it is a work in progress.
- I added ExampleApplication.h and ExampleFrameListener.h to the distribution. I really didn't know any other way to make certain they were available. They are not part of the standard ogre headers.
- autogen.sh doesn't remember it's executable bit and I wouldn't know how to add it in bzr. You can still just run it with sh autogen.sh so not a big problem.

The diff can be downloaded here.
http://wina2.ugent.be/~karel/patch.diff

This patches provides the bare minimum to make it a proper gnu Makefile project. It compiles ETL as a dynamic library and the Sample as a binary. It also creates an etl-0.1.pc for pkgconfig for easy inclusion in other projects.

Instructions:

Install the necessary dependencies. (OGRE >= 1.4.0/OIS >= 1.0/CEGUI >= 0.5.0). On ubuntu gutsy this would be.

sudo apt-get install libceguiogre-dev libcegui-mk2-dev libogre-dev libois-dev

But your distribution may vary.

Now to install ETL v3 use:

sh autogen.sh
make
sudo make install

to configure/compile/and install the plugin.

Well that was the easy part. Now I wasn't exactly certain how the Sample was supposed to run. Reading the source files I got the necessary media and cfg files from ETL-2.2 and the ogre sources but didn't get further than this.

http://wina2.ugent.be/~karel/ETL.png

which is obviously missing textures. I didn't really look thoroughly through the code yet but figured I'd ask first which config files and media files you use to run your demo exactly.

CABAListic

30-05-2008 07:22:49

Thanks for that, I'll check it out some time.
Unfortunately, development of ETL is currently more or less frozen (as you may have noticed) as I currently lack the time, having begun my bachelor thesis. Might be that I'll find some time earlier, but don't expect development to continue before mid of August.

MOROHEUZ

02-06-2008 14:02:39

This is sad, especially now that Shoggoth is at start.

A small consolation is perhaps SPT2, but for much larger terrain and is written for smaller amounts of data

CABAListic

02-06-2008 15:32:21

If you're worrying about Shoggoth, just add that to the ETMv2 maintenance thread

MadLion

17-06-2008 23:27:26

Well that was the easy part. Now I wasn't exactly certain how the Sample was supposed to run. Reading the source files I got the necessary media and cfg files from ETL-2.2 and the ogre sources but didn't get further than this.

http://wina2.ugent.be/~karel/ETL.png

which is obviously missing textures. I didn't really look thoroughly through the code yet but figured I'd ask first which config files and media files you use to run your demo exactly.

Same here but i'm investigate on this.
There are several problems...

ETSample.cpp

After change splating (splat method) or change colormap (paint method) no update is called so textures are not updated. Just add the following:

line 153 after mSplatting->splat(paintPos, mEditBrush, mChosenTexture, brushIntensity);
mSplatting->update();

line 164 after mColourMap->paint(paintPos, mEditBrush, mPaintCol, brushIntensity);
mColourMap->update();

Next is the material name... see line 535
MaterialPtr material (MaterialManager::getSingleton().getByName("ETTest"));
In ETM 2.2 the material is called "ETTerrainMaterial". So change this.
MaterialPtr material (MaterialManager::getSingleton().getByName("ETTerrainMaterial"));

But now you get all black, because there is a problem with lightning.
A workaroun is to comment the lightning pass in ETTerrain.material.

Now you should see the textures without lightning.

I hope this helps.

MadLion

17-06-2008 23:48:53

Ok i got it! ^^

Replace line 529-530 with
mLightMap = new ET::DynamicTexture("ETLightmap", "ET", 128, 128, PF_BYTE_RGB, Vector2(10,10), Ogre::ColourValue(1,1,1));

You must have a member
ET::DynamicTexture* mLightMap;

Than add the following to updateLightmap()

line 445



ET::ColourArray2D lightMap = ET::createLightmap(mTerrain, 128, 128, Ogre::Vector3(0,-1,1), Ogre::ColourValue(1,1,1), Ogre::ColourValue(0.4,0.4,0.4));

mLightMap->setColourArray(lightMap);

mLightMap->update();

That's it!
Hope it works for all.

CABAListic

14-07-2008 18:12:20

I've not posted an update for quite some time, and unfortunately I don't have one. Still, I wanted to keep you informed on my current time plan.

At the moment, I'm a bit occupied with my bachelor thesis and one final exam for this term. Once the exam's done with (will be this Friday), I should be able to find some time again for ETL. First thing I'm going to do then is to switch the version control system to SVN. While I do like bazaar, I understand that it's an obstacle for most people here. In addition, this will allow me to install the Trac wiki system on my server which will be the future place for ETL to be filled with examples, code snippets, advanced documentation, i. e. all the things ETM currently lacks in my opinion.
As for coding, I thought about tenttu's comments a bit more and will make some small refactorisations to address some of his concerns. But the top priority is to get a decent build system done, so I guess I'll have to dive into CMake.

CABAListic

20-07-2008 16:38:40

Alright, ETL has a 'new' home at http://etl.oddbeat.de . Not much to see there, yet, but it's going to be the place to find tutorials, code snippets, ... and the bug tracker for the project. Also, ETL has now moved to a subversion repository which you can checkout via:

svn co svn://oddbeat.de/etl/trunk etl

There is now the beginnings of a CMake based build system which can build the library (though I have to add some more sophisticated means to find Ogre, currently you'll probably have to provide the paths yourself). The sample is not currently included, I'm going to rewrite it.

BloodyFanatic

20-07-2008 22:59:39

may i ask how "mature" etl v3 already is? e.g. what new features are already implemented and what are not?

CABAListic

21-07-2008 08:51:07

The features of ETM v2 are *mostly* there. Shadowmap generation is still missing which I'm planning to address soon. However, I'm not yet promising interface stability, in fact, some interfaces are about to change slightly.
I might have more to say at the weekend, I'm currently rewriting the demo, with that done it should be easier to check out ETL and see where it is.

CABAListic

23-07-2008 20:23:34

The last few days I've been at university till late evening hours, so that's left me with little time for ETL. But thankfully that'll be no issue next week.

Anyway, jjp created a logo for ETL *drumroll*:

CABAListic

11-08-2008 11:23:21

Some of you might have noticed my absence in the last days. Just when I was finally ready to do some more serious work for ETL, I got sick and, somewhat sudden, I was sent to hospital from which I've only just returned. While I'm feeling fine, I'm not allowed to sit (properly, anyway) for a few more days, so I'm still somewhat limited in regards to coding.

Sorry for that. I'll keep you posted when I'm back in business

kungfoomasta

11-08-2008 17:15:00

Get better soon! Get fully recovered first and then work on ETL.

syedhs

11-08-2008 19:30:46

Get back well soon... we just cant wait to lay our paws on your new ETL :wink:

Deathbrush

24-08-2008 12:08:51

Get healthy first, no matter what programmers often come to believe
code != life
I know, very old joke, still hard to resist.
And thanks for the amazing tool.

pra

31-08-2008 20:47:44

I have a question about ETL3: Somewhere you said that it will be possible to make a paging solution with it, by having multiple heightmaps which are connected by the terrain manager. What happens if I take my brush and try to deform and/or paint texture exactly at the seam where two heightmaps are connected?
Also, will it be possible to have different sets of textures for each page? If yes, and if I can paint textures on multiple pages by clicking at the border, can I do something to paint the same texture on all of them? (assuming they share this particular texture)

and an OT question: is that you?
http://german-bash.org/4359
http://german-bash.org/55918
:lol:

CABAListic

31-08-2008 21:05:30

I have a question about ETL3: Somewhere you said that it will be possible to make a paging solution with it, by having multiple heightmaps which are connected by the terrain manager. What happens if I take my brush and try to deform and/or paint texture exactly at the seam where two heightmaps are connected?
Also, will it be possible to have different sets of textures for each page? If yes, and if I can paint textures on multiple pages by clicking at the border, can I do something to paint the same texture on all of them? (assuming they share this particular texture)

Well, that was kind of the point: You *can* build a paging solution, but it doesn't come out of the box (i. e. paging is not an ETL feature)

From the point of ETL, pages are independent from each other, there is no manager which connects them. The part about connecting them only concerns their visual representation, i. e. you can tell them about adjoining pages so that they can synchronise their LOD levels to avoid cracks at the borders (that's the feature ETM is currently lacking). Other than that, you have to take care of all the management (read: build the paging manager yourself

). If you apply a brush to one of the pages, it will not automatically affect neighbouring pages. But since you can now apply the brushes to actual Ogre coordinates, you could just apply the brush to all active pages with very minor overhead for pages which are not affected.
Due to the pages being independent of each other, they can naturally have differing texture sets. However, you would again have to take responsibility for making them match at the borders. As for painting them, if my advanced splatting manager is going to work they way I envision it, then you would tell the splatting instance for each page which texture you want to paint by name, and it would map this to the internal texture index (which can and would probably differ between pages).

Re OT: Yeah, that's me, though I have not the slightest idea who submitted the quotes. The second one isn't even all that funny

pra

01-09-2008 02:26:27

By "connecting" i meant the visual representation part^^

Hm ok, that sounds good, i think i can do it.
Btw, could I theoretically make a box with all 6 sides being terrain? Or, could I attach a heightmap to a node and move it around?
But, tell me more about the splatting system: What do I have to do, if I have a list of x textures (and maybe y coverage maps, too), and I want it to use them? does it dynamically create it's material? What about the shader? Can I set how many channels I want to use? Is it capable of dynamically remove some of the textures and updating the covmap?
And, can I make parts of the heightmap transparent? I'm alerady thinking of a way to make holes into the OgreNewt::Body that represents the terrain physically, I almost think making it look like a hole would actually be more difficult. And you also said I can directly access the vertices? that would definitely make things easier^^

@OT: good to know you can speak german, now I can annoy you through PM if something is so complicated that it is beyond my english skill^^

CABAListic

01-09-2008 08:18:41

Like a cube, you mean? Theoretically yes, but this will only work if all borders are flat, otherwise there *will* be cracks. You can attach ETL objects to nodes and update their position by moving the node.

As for the splatting system: At the moment, there's only the basic SplattingLayout which is, feature-wise, nearly identical to the current SplattingManager in ETM. I do have plans for a more advanced solution which manages the textures internally; look back in this thread, I think I described it before. But it's supposed to work like this: You create a SplattingManager with a given texture format (which specifies the number of channels in the coverage maps). Then, whenever you want to paint, you pass the *name* of the texture you want to use, and the system will internally look if it's already painted that texture before. If not, it finds a free space in the texture index map, potentially creating new coverage maps. There will be listener hooks to be notified about texture changes so that you can update your material accordingly. What support ETL will bring for materials and shaders, though, is not yet certain. If in doubt, to properly fit your environment, you have to manage your materials and shaders yourself, anyway.

bymulker

04-10-2008 13:10:24

congratulations to Cabalistic for such a great editor tool AND get well soon

i've integrated ver.2.3.1 to my editor but i failed making it work with more than 6 textures although i've followed most of the threads here. My card supports 2.0 at vs and ps. i tried to put the other "6" texture at another pass but i failed at organizing the shader code accordingly. As i understand the dynamic texture at ver. 3.0 will address the problem. Could someone please give the initialization code to achieve the photos Cabalistic have posted at page 2 of this threaad? i mean the minimal code and shader just to make version 3.0 work?

CABAListic

04-10-2008 14:36:21

No, you misunderstood that. Texturing the terrain is ultimately a material task, and the material names the splatting textures to be used. Even with ETL v3 you will still have to organise your textures into passes accordingly. I do plan to include a class which will help with that, but it's not yet been completed. The DynamicTexture simply allows to paint on a texture and is used as the new basis for the editable splatting part.

bymulker

04-10-2008 16:49:23

oh ok .. then thanks for the reply and we'll stay tuned

btw wouldnt it be better to do splatting in-memory without any needs to tweak shaders?

CABAListic

04-10-2008 18:58:33

oh ok .. then thanks for the reply and we'll stay tuned

btw wouldnt it be better to do splatting in-memory without any needs to tweak shaders?
What do you mean with in-memory? The splatting is done in real-time on the graphics card with the help of shaders. There is no way to do it without shaders, so at some point you must deal with the shaders.

If you mean editing the material file, then sure, you can create and manage the material in code. In fact, for an editor application, you will have to do that. But ETM/ETL doesn't really care about the terrain material, it's your job to do

ETL will probably offer help for the shader setup as I said, but nothing more.

CABAListic

10-10-2008 02:07:24

I have started writing a shader generator for splatting shaders. This is primarily targeted for internal use by my generic splatting material class, but it could be used directly if desired. The purpose of the generator is to generate shaders for various shader models (1.1, 1.4 and 2.0 currently) which fit exactly on the requirements of a specific material pass. That way any unnecessary operations in the shader are avoided. For example, if you're using 4 channel splatting maps, but you currently only use the first two channels of a map, then the generator can produce a special shader which will only process those two first channels and thus save the calculations on the last two.

I don't know how much this will improve performance if at all (probably depending on the graphics card), but the generator can also create shaders for odd cases. Let's suppose you are using two 4-channel splattings maps (hereafter named map1, map2) and 7 splatting textures (hereafter named tex1, ..., tex7). And suppose your graphics card can only handle shader model 1.1, which only supports four texture units. Then the generator will create special shaders so that the process can be split into 3 passes like this:

Pass 1: Uses the first 3 channels of map1 to splat tex1, tex2, tex3. [/*:m]
Pass 2: Uses the last channel of map1 and the first channel of map2 to splat tex4, tex5.[/*:m]
Pass 3: Uses the two middle channels of map2 to splat tex6, tex7.[/*:m][/list:u]
3-channel maps are still better suited for sm1.1 fallbacks, but at least it'll work.

pra

10-10-2008 15:26:17

what about dynamic lighting and fog?
from what I understood, ATM you need a custom shader to make these things possible

CABAListic

10-10-2008 15:46:14

The generator only creates shaders for the splatting passes of the material. Dynamic lighting belongs in a separate pass and has therefore nothing to do with the generator. As for fog, I frankly don't know how fog works in shaders, but would assume that this can also be done in a separate pass after the splatting.
Anyway, the splatting material class I'm writing only takes responsibility for splatting. You have to do everything else. This is done by providing a base template material which has all the other stuff in it you need. The splatting material clones this base material and adds the splatting passes at the beginning of the best available technique as necessary.

kungfoomasta

10-10-2008 18:44:37

We should let CABAListic focus on the splatting implementation first, its not trivial work and many people want it.

Supporting all the different shaders must be a pain. Worst case, just skip 1.1 and work with 3 or higher. :lol:

jjp

10-10-2008 21:31:26

Nah, this will try to make the most out of every gfx card. Something for every model from 1.1 up to 3

CABAListic

10-10-2008 21:50:38

I just finished the first version of the SplattingMaterial class (alongside with the ShaderGenerator). It's basically working, but I'm expecting a lot of border cases to debug

Anyway, I'll now focus on creating the advanced splatting manager which will allow me to test the splatting material indepth.

If you're interested in the interface, look at the source:
http://etl.oddbeat.de/trac/browser/trun ... Material.h

kungfoomasta

10-10-2008 22:19:28

Cool idea, posting up the header so I can read it at work. :lol:



72     /**  

73      * Specify which texture coord set to use for the splatting maps. 

74      * If you intend to apply this material to a single terrain patch, 

75      * then enable this. 

76      */ 

77     void setUsePatchLocalTexCoords(bool patchLocal);

This description doesn't seem accurate, I'm not specifying any texture coord set from what I see.

Can you elaborate how this works, how texture coords need to be different if applying to individual patches vs the entire terrain?

Also I see we have public constructor/destructor, which means we have to manage this ourselves. Can we assume that the material is unregistered from MaterialManager on destruction?

For the most part the interface is simple and looks very welcoming.

CABAListic

10-10-2008 22:49:36

Cool idea, posting up the header so I can read it at work. :lol:



72     /**  

73      * Specify which texture coord set to use for the splatting maps. 

74      * If you intend to apply this material to a single terrain patch, 

75      * then enable this. 

76      */ 

77     void setUsePatchLocalTexCoords(bool patchLocal);

This description doesn't seem accurate, I'm not specifying any texture coord set from what I see.

Can you elaborate how this works, how texture coords need to be different if applying to individual patches vs the entire terrain?
Agreed, it's not exactly well worded. What I mean is that every patch has two texture coord sets. The first, primary one is the "global" texture coord for the entire terrain (well, Page, that is), while the secondary one is the "local" texture coord set. The local one means that the lower left vertex of the patch always has the texcoords (0,0) while the upper right vertex always has (1,1). So drawing texture in relation to this set will mean you draw the texture relative to the patch dimensions. On the other hand, the global set has texcoord mappings according to the subset it represents from the entire terrain (Page).
So, normally you will be using the global one, *but* if you intend to specialise splatting for each patch in order to save a few operations (rare textures which are used in only few places don't need to be drawn on the whole terrain) you will need the local set for the specialised splatting maps, because they would otherwise be stretched across the whole terrain incorrectly.

Also I see we have public constructor/destructor, which means we have to manage this ourselves. Can we assume that the material is unregistered from MaterialManager on destruction?
Yes, you can. Unless I make coding mistakes, the classes always clean up what they create

For the most part the interface is simple and looks very welcoming.

If you wanted to add some convenience methods, I would suggest ability to write the material to file, and being able to change the resource group after creation. Also, are we able to call createMaterial multiple times without problem, or is this class meant to be a 3 part deal: setup, create, use
Technically, you can already write the material to file by use of the MaterialSerializer, just get the created material via getMaterial and pass it to the Serializer. I don't think it would add much to replicate that functionality in a member function. As for changing the resource group, this would require a recreation of the material which would be possible in principle, but means some overhead, and problems if it's in use. I'm not sure if it's really that important, after all, how often do you move your materials between resource groups?
And finally, yes, you can absolutely recreate the material. Change the settings and call createMaterial again. If it couldn't do that, it would be worthless for my advanced SplattingManager

kungfoomasta

10-10-2008 23:36:00

Awesome, sounds good. ETL is gonna rock.

Allowing changing of resource groups after creation doesn't seem worthwhile, after reading your comments. I knew about the Material Serializer, so its not a big deal, thats why its a convenience function.

CABAListic

20-10-2008 20:22:35

So, I'm currently working on the advanced splatting manager. It can already dynamically add textures, so I'm working on the part where it removes unused (or requested) textures. However, this requires some slight structural reworks inside ETL which takes some time. I'm designing a base class for all editable objects and also need to rework the Brushes a bit. However, once this is done and functional, ETL has taken a big step towards beta stage, so bear with me here

ealogar

22-10-2008 10:58:28

Great job !!
Waiting for new release ...

ppClarity

23-10-2008 16:15:24

I'm evaluating using ETL in a paging situation and have been inspecting your code. So far it looks great (and clean).

When a page needs to be evicted, how should it inform the neighbors? It doesn't look like the destructors for either ETPage or ETPatch does anything there. I'm guessing that should be done explicitly in ETPage's destructor.

CABAListic

23-10-2008 16:23:17

You mean inform the registered neighbours that it no longer exists? Yes, the destructors should do that, but it's possible I missed it. I'll try to remember putting it in

Once I get the first beta release out, I'll also introduce the official bugtracker, so if I forget, you can file a report then, and I'll definitely sort it out for final release *g*

ppClarity

23-10-2008 16:49:02

awesome and no rush, if that part is missing, extending the code to support it doesn't seem to be that big a deal.

I hope I can make some time over the weekend to play with ETL, it looks like it will fit very well with my project.

CABAListic

24-10-2008 20:47:08

The SplattingManager is now working, mostly. It automatically adds new textures and can also detect unused channels (i. e. those you have used earlier, but overwritten with other textures). Whenever you splat with a texture the manager doesn't know yet, it checks all current channels whether they are in use (done with a dirty rectangle technique, this is very fast) and either reuses an unused one or adds the new texture as a new channel. Via a listener you can automatically inform the SplattingMaterial of those changes, and voila, you have a completely dynamic terrain splatting system which takes care of itself.

It's not perfect, yet, but the foundation is working, the rest is code cleanup

I'll now move to the matter of creating maps of various sorts for the terrain (lightmap, normal map, shadow map, ...) and then get the undo/redo functionality in.

kungfoomasta

24-10-2008 23:15:54

Awesome! Can't wait until I can get to use this. (I'm blocking myself right now

)

ppClarity

25-10-2008 19:22:31

I've got two ETL's next to each other and called setNeighbour() to connect them. It's segfaulting in IndexHandler::stitchBorder() around line 285. Unless I'm missing something the loop's end condition will never be met.

(gdb) print k
$33 = -1258794
(gdb) print halfsuperstep
$34 = 0
(gdb) print superstep
$35 = -1
(gdb) print step
$36 = -2
(gdb) print hiLOD
$37 = 1
(gdb) print loLOD
$38 = 158775008

Starting around line# 276

      for (k = halfsuperstep; k != superstep; k += step)

      {

        size_t jk = j + k;

        if (j != endx - superstep || k != superstep - step || !omitLastTri)

        {

          if (horizontal)

          {

            *pIdx++ = index(j+superstep, starty);

            *pIdx++ = index(jk, starty+rowstep);

            *pIdx++ = index(jk+step, starty+rowstep);

          }

          else

          {

            *pIdx++ = index(starty, j+superstep);

            *pIdx++ = index(starty+rowstep, jk);

            *pIdx++ = index(starty+rowstep, jk+step);

          }

          numIndexes += 3;

        }

      }

CABAListic

25-10-2008 19:38:33

The stitching code is copied from Ogre's TSM and has also operated in ETM v2. So far it's never failed, so I assume that it is correct. More likely the neighbouring setup is incorrect internally, and the algorithm gets bogus data on the LOD level of the supposed neighbour. Your posted values for loLOD support that theory. I'll investigate.

ppClarity

25-10-2008 22:00:50

Yeah, the stitching works for the page's own patches, it only blows up in the page neighboring.

Both are 257x257
First one is positioned at (-128,0,-128)
Second is at (-128,0,128)

first->setNeighbour(ET::DIR_SOUTH,second);

Or should that be DIR_NORTH?

CABAListic

25-10-2008 22:13:48

No, should be fine. Even if it were the wrong direction, it would just connect the wrong patches, and since the patches don't care about each other's position, that should not result in the crash. Perhaps the neighbouring setup by the Page is incorrect.

Edit: On second thought, I'm not so sure about the direction. I always get my layout confused

Try north if you haven't already. But anyhow, it should result in a crash, even if you get that one wrong.

CABAListic

25-10-2008 23:06:08

I just tried to reproduce this, I didn't succeed. I created a second page and set it neighbouring the first, but nothing bad happens. Can you show some more of your setup code, please?

BTW: I just fixed the issue with the Patches not unregistering themselves from their neighbours, at least I think so.

ppClarity

26-10-2008 18:32:22

I poked around a bit more and discovered that the LOD's were insane on the 2nd page. This is happening because the LOD's aren't initialized until the page becomes visible and mine just happened to be created behind the camera. I'll adjust the code and retest in a few hours (have some errands to deal with).

ppClarity

26-10-2008 20:16:11

Adding mCurLOD = mMaxLOD - 1 in ET::Patch's constructor cleared up my issue. No idea if that's any more or less correct than mCurLOD = 0 but I guess anything less than mMaxLOD will work.

CABAListic

26-10-2008 22:03:27

Ah right, that explains it. I'll add the fix, though for this case an initialisation to 0 is probably more sensible because in that case no stitching is needed at the affected borders (stitching works from higher to lower detail, i.e. stitching is needed if the level of the adjoining patch is higher (thus lower on detail).

kungfoomasta

05-11-2008 00:21:39

According to the API of that latest release, (I'm at work) I could not find any methods for specifying the render queue used to render terrain. I would like to request a feature for an API allowing us to set the Render Group the terrain is drawn on.

For background information, I'm trying to place quads under units and draw a ring to imply a unit has been selected. When I modify the material of the quad to enable transparency of the texture unit of the pass, the object is grouped behind the terrain, so the decal never appears.

CABAListic

05-11-2008 02:16:17

I'll look into it. Shouldn't be too difficult.

kungfoomasta

08-11-2008 19:24:53

I also want to request a function to set the visibilityFlags of the terrain. If each Patch is a MovableObject than this the bulk of this functionality is already supported, assuming we have access to Patches. Having a convenience function to iterate through patches and set their visibility mask would be a good addition also.

And.. any updates?

CABAListic

08-11-2008 22:48:24

I'm unfortunately a bit busy right now as I have some university business ahead. I'll have to see where I can steal some time for ETL :-/
Anyway, Patches are MovableObjects and should have the visibility functionality built-in. You will be able to access the Patches directly via the Page. Convenience functions to set some properties on all patches could be implemented on the Page. They're probably a good idea, but currently not high priority

CABAListic

10-11-2008 20:55:55

I managed to do a small update today: I moved the map generation code to a separate file and wrote some blur filters you can use on the generated maps. Available right now is a normal map and light map generation (without shadows), and you can optionally apply a box or gaussian blur filter to the generated maps. The filters are written generically, they can operate on ET::Array2D of any type so long as the type supports addition and division by a scalar. So you could even use them to filter a height map, if you like

In ETM a box filter is hardcoded in the shadow map generator. Now you have the freedom to use it whenever you like and on whatever map you need it.

Example usage:



ET::ColourArray2D lightmap;

ET::generateLightMap(lightmap, mTerrain, ...);

ET::gaussFilter(lightmap, 3, 0.867);

3 means the kernel radius, i. e. examine 3 additional pixels to either side of the current pixel. 0.867 is the sigma of the gaussian distribution used to calculate the kernel matrix.

CABAListic

11-11-2008 21:07:13

I`ve added shadow map generation today. I even managed to get rid of a couple of lines of the (terribly long) algorithm. The results are not as good as with ETM yet, though, which is probably mainly due to the fact that terrain heights are not interpolated. I will accomodate that.

It also seems to me that the generation is slightly slower than with ETM, at least the generation is noticeable in the demo editor. But that may be due to a number of things, I`ll have to double check that. If there is a bottleneck in comparison to ETM, I`ll find it

pra

12-11-2008 09:43:59

what if I have 2 terrains connected, i.e. if i want to make a paging solution? will a mountain on the edge cast a shadow to the neighbour page?

CABAListic

12-11-2008 10:17:20

The map generation works on exactly one Terrain object. But the terrain object is not the graphical representation, in principle you can build a terrain which is larger than a single page, so you could generate a shadow map for more than one page. However, you will have to do some work to get the desired results. Only non-paging terrain will immediately get a nice shadow map

pra

12-11-2008 21:54:44

hmm...
Can I make ETL generate a shadow map without actually rendering the entire terrain?
I think I want to generate it once, and then cut it to pieces according to the heightmap pages. Is that possible?

CABAListic

12-11-2008 21:58:44

That's possible. As I said, the terrain object has nothing to do with its graphical representation.

pra

15-11-2008 15:24:02

and yet another thing about the shadow map:
if the terrain is large enough, it can take a while to recalculate the shadowmap, even if you only edited a little. I understand that it would be difficult to update the edited part only, because high mountains could cast long shadows onto this area, or a deep canyon could make the shadows of the edited area too long. I suppose "light map generation (without shadows)" means it only makes one side of a mountain darker, but without making it cast shadows? I believe it would be better to do only that during terrain editing, is it possible to recalculate that for the edited part only?

CABAListic

15-11-2008 17:53:28

In principle, yes, since the value of each pixel in the lightmap is basically just the scalar product of light vector and terrain normal at that point. Therefore it doesn't depend on any other pixel. But the functions do not support specifying subranges and I'm reluctant to add that because it complicates the function signatures and their implementation - and since in 99% of the cases you'll probably not need this, I don't think it's justified.

In any case, as already pointed out, ETL is not designed for large terrains. For a lightmap of up to 512x512 pixels I think lightmap calculation should not be an issue since the mathematical operation is relatively simple, so doing it on 262144 pixels shouldn't be noticeable. Anything above that will soon need paging, anyway, in which case you would have the lightmap divided just as well.

Buckcherry

22-11-2008 20:35:33

Hi everybody,

I have been reading this thread and I am a bit confused.

I though ETM was not implementing tile paging for large terrains.

Is it supported now??? It would be a very good news!

kungfoomasta

22-11-2008 23:52:38

I think its flexible enough to allow paging with some work, but not supported by the author. (CABAListic)

CABAListic, I think you tossed me an SVN link a long time ago, is it possible you can post it up again, or make it a sticky? (Assuming its mature enough for use) I just finished my MeshViewer, and my SceneEditor is next. I want begin with the latest version of ETL, especially to make use of the built in splatting capabilities.

Buckcherry

23-11-2008 10:36:39

In this thread there are people talking about connecting 2 maps... So, maybe there is a newer version of ETL.

Where could I download the lastest version of ETL?

calsmurf2904

23-11-2008 11:39:22

Buckcherry

23-11-2008 17:00:09

It can be compiled using Visual Studio 2008?.

I have been taking a look into the SVN repository and I can't see any solution file.

calsmurf2904

23-11-2008 17:38:45

need to create them yourself...I have succesfully compiled it + demo with visual C++ 2008 with SP1

kungfoomasta

23-11-2008 19:29:26

calsmurf2904, thanks for the link.

kungfoomasta

24-11-2008 01:19:14

Tortoise SVN gives the following:



Command: Checkout from http://etl.oddbeat.de/trac/browser, revision HEAD, Fully recursive, Externals included  

Error: XML data was not well-formed  

Finished!:

Same error results when trying to view the repo.

calsmurf2904

24-11-2008 06:21:19

no...its the http link...only internet explorer/firefox etc. can view it.
I am currently searching for the svn link...hold on

Edit:
Alright, ETL has a 'new' home at http://etl.oddbeat.de . Not much to see there, yet, but it's going to be the place to find tutorials, code snippets, ... and the bug tracker for the project. Also, ETL has now moved to a subversion repository which you can checkout via:

svn co svn://oddbeat.de/etl/trunk etl

There is now the beginnings of a CMake based build system which can build the library (though I have to add some more sophisticated means to find Ogre, currently you'll probably have to provide the paths yourself). The sample is not currently included, I'm going to rewrite it.
Got it

kungfoomasta

24-11-2008 08:03:39

I'm able to browse in IE using the first link, but the other links don't work in SVN. Fortunately I found a link at the bottom that allowed me to download a zip of the entire trunk directory.

alextreme

25-11-2008 17:54:19

Hey CABAListic,

I've been playing around with your ETL "sample", thinking it might be useful for making terrains for an Ogre project of mine. Alas, I only found out saving/loading wasn't working after I had made myself a perfect terrain

Anyway, I figured it was good enough to work on, so I added saving/loading myself. You can find the patch against the latest SVN version here: http://alextreme.org/etl_saveload.patch

Saving is done by entering a directory name, with the height/colour/lightmaps being stored in it. Loading naturally works the other way around, by entering a directory name with height/colour/lightmap images (PNGs). If it sounds similar to one of your earlier post, this is not a coincidence

I've tried to conform as much as possible to your coding style, but please look it over as my C++ is rather rusty. This patch is very much a work-in-progress, as I've yet to add splatting to the save/loading functions and I've only tested it in Linux/Debian. YMMV, feedback is welcome.

If you don't mind me lending a hand, let me know and I'd gladly help you flesh out this project further. I think it's at a state where many Ogre projects could benefit from it.

CABAListic

25-11-2008 17:59:44

Thanks for your work. I'll review it and potentially add it to the library, but I'm currently lacking time (again). I'll get back to ETL in two weeks to hopefully finish it finally. In the meantime I would *not* encourage you to try to add save/load for splatting, because the structure of the splatting manager might still change slightly

alextreme

25-11-2008 18:05:40

Cool, and thanks for the quick reply!

I'll hold off on the splatting until you're able to review the patch. I'll make some pretty terrains in the meantime

kungfoomasta

25-11-2008 18:41:01

When you say saving/loading color maps, what are these color maps used for? In my mind color maps are what define splatting, the RGBA channels mean each color map can manage 4 splatting textures. Are these color maps you mention not for splatting?

alextreme

25-11-2008 20:21:05

I've placed a few images online to illustrate on http://alextreme.org/gfx/etl/

The screenshot shows the terrain in the ETL editor.
- heightmap.png shows the height of the terrain
- lightmap.png defines the shadows
- colourmap.png shows colouring done using the colour/paint brush (see the panel to the right in the screenshot, below texture). The colourmap is a RGB image overlayed over the textures of the terrain (if I understand correctly).

Simply put, splatting is drawing textures on the terrain, while painting lets you add colour on top of those textures

alextreme

05-12-2008 12:23:17

I've been playing around some more with ETSample. This time, I've also added support for creating new terrains with an arbitrary size via menu->file->new.

Patch can be found here: http://alextreme.org/etl_saveloadnew.patch

The problem with larger terrains (I was trying one of 513 units) is that calculating the lightmap takes quite long, about 10 seconds. I've moved this bit of code to a new button in the interface instead, so that the lightmap isn't calculated every time you modify the terrain but only when you want to.

It isn't a good solution. I'll look into optimizing this bit of etl when I have the time, but a quick glance at the code showed no easy optimizations. It was an O(n^2) algorithm afaik, so that this becomes a problem when moving from 129 to 513 sized terrains isn't strange. Maybe a bit of profiling will show the bottleneck...

CABAListic

05-12-2008 16:08:48

First of all, you could try if the "old" ETM v2 routine can cope with 513x513 terrain. The algorithm is identical, but the implementation is a bit different, plus there are currently some temporary steps involved in getting the calculated lightmap to an actual texture which can be avoided. Therefore, if ETM can cope with that size, ETL will be able to in the future, since I'm going to eliminate the mentioned problems in any case (therefore, no work on your side needed

). If not, then that's simply the limit of the algorithm. In that case, the shadow map probably needs to be skipped.

Or, you could try and still use a 256x256 lightmap even for a 513x513 terrain. The calculation cost depends on the size of the lightmap. Of course, it might be that it doesn't look good enough, but then again, it's just needed as a preview in the editor, a proper lightmap could be calculated when saving terrain.

Pieron

16-12-2008 19:00:44

hello all.

I'm want to make paging solution with current development version of ETL. I'm downloaded sources and look to code. As i understanding, i'm need to:

1) Create ETFlatTerrain and ETPage instances for every page in terrain
2) Call ETPage::setNeighbour for neighbour terrain pages
3) Create custom terrain manager, which handle pages coordinates and calling functions for properly ETFlatTerrain/ETPage instances

But i don't understading following:
1) Do i need to calculate correct LOD for neighbour pages ? (when camera updated) Or its calculated automatically by GPU program (GeoMorhping), so i don't needed in any additional actions?

2) How i can get to know which page is need to be loaded ? (when camera is moving, for dynamically loading heightmaps from disk). I think, i need to calculate camera position in every frame and load/unload properly heightmaps (and create/destory ET instances). But how? I'm tried to look in the SimplePagedTerrain sources, but without results

Sorry for my poor english

Thanks for answers

CABAListic

16-12-2008 21:07:56

1) No, you don't. It's already handled by the Pages.

2) Well, that's the difficulty of a paging solution. I'm afraid you're on your own here, I'm staying out of any paging terrain algorithms because they are difficult enough to get right, and I don't need them at the moment

pra

16-12-2008 23:13:15

hm, long time ago, I tried to make a general paging solution (for loading/unloading meshes). I don't remember why I stopped, but I think the part what detected which pages to load/unload worked alerady...
Warning: this code is old, and I'm not really sure what I did back then, but i'm pretty sure it worked, but i cannot guarantee that...

The problem I had was to store which pages I loaded and which not. First, I wanted to make a 2d vector what also allows negative values for indices, then to make a std::map<vector2d>, and then i decided 9 pages are enough, and made an array with length 9 of a special struct:

struct Page

	{

		Page()

		{

			x=0;

			z=0;

			loaded=false;

		}

		int x;

		int z;

		bool loaded;

	};

Page mLoadedPages[9];



bool Level::isPageNeighbour(int main_x, int main_z,int x, int z)

{

//is the page at (main_x,main_z) neighbour of (x,z)?

	if((main_x == x+1 || main_x == x || main_x == x-1) && (main_z == z+1 || main_z == z || main_z == z-1))

		return true;

	return false;

}



void Level::updatePaging()

{

	//this function was called each frame, to look what pages must be loaded or unloaded

	Page noLoad[9];

	int endIndex = -1;

	

    //this part looks which pages to unload, and which to keep

	for(int i=0;i<9;i++)

	{

		Page cur = mLoadedPages[i];

		if(!isPageNeighbour(curPage_x,curPage_z,cur.x,cur.z))

		{

			if(cur.loaded)

			{

			//it is loaded, but it must not

				unloadPage(cur.x,cur.z); //loadPage and unloadPage did nothing but setting the variables yet...

				

			}

					

			

		}

		else if(cur.loaded)

		{

			//it should stay loaded

			endIndex++;

			noLoad[endIndex] = cur; // to prevent double loading. hm, loadPage could look if a page alerady exists itself, instead of that

			

		}

	}

    //this part now looks which pages to load

	for(int x=0;x<3;x++)

	{

		for(int z=0;z<3;z++)

		{

			bool mustLoad = true;

			int curX = x+curPage_x-1;

			int curZ = z+curPage_z-1;

			for(int i=0;i<=endIndex;i++)

			{

				if(noLoad[i].x == curX && noLoad[i].z == curZ)

				{

					mustLoad = false;

					break;

				}

			}

			if(mustLoad)

			{			

				loadPage(curX,curZ);				

			}

		}

	}

	

	

}



void Level::getPlayerPage(int &x, int &z,bool useBorders)

{

    //now this gets the page coordinates of the player

    //useBorders is a stupid thing, i think i only made this optional for testing purposes

    //the idea is: if the player went just over the paging border, and just after the pages loaded, he makes accidentaly a 

    //step back, he will again trigger a loading sequence, and then another one, when he wants to correct his mistake.

    //so, the borders have a certain thickness, and pages are loaded when you leave the border

	Vector3 playerPos = player->getPosition()-pageStartOffset;//pageStartOffset was supposed to be the position of the paging origin in world coordinates. not sure why i did this...

  /*            /#\

     |     |     #     |     |     |   

	-+-----+-----#-----+-----+-----+---

	 |     |     #0/+1 |     |     |

	-+-----+-----#-----+-----+-----+---

	 |     |-1/0 #  0  |+1/0 |     |   X

	-+-----+-----X==========================>

	 |     |     |0/-1 |     |     |

    -+-----+-----+-----+-----+-----+---

	 |     |     |     |     |     |

	*/

	//(x-a/2)/a = x/a - 0.5

	Real relX = playerPos.x / pageSize + 0.5; //X-position in Pages

	Real relZ = playerPos.z / pageSize + 0.5;



	Real actualX = Math::Floor( relX ); //actual page coordinates

	Real actualZ = Math::Floor( relZ );

	if(!useBorders)

	{

		x = actualX;

		z = actualZ;

		return;

	}



	

    //pagingBorder = 0.3;

	if(actualX + pagingBorder < relX && actualX+1 - pagingBorder > relX)

	{

        //ok

		x = actualX;		

	}

	else

	{

		//X is still in the border

		x = curPage_x;

	}

	if(actualZ + pagingBorder < relZ && actualZ+1 - pagingBorder > relZ)

	{

		z = actualZ;		

	}

	else

	{

		z = curPage_z;

	}

	





}

I hope this helps somehow, i'm too lazy to look at it right now..
I'm very interested in a paging ETL, but again, lazyness...
and I also do not want to port my program to a ETL that isn't ready yet...

Pieron

17-12-2008 21:10:40

Thank for answers

2pra:

As i understand, your approach is just loading neighbour pages? I think this is right for large pages.
But for small pages this is unnacceptable(when the clip far distance for camera large).

May be calculating pages based on camera clip distance will be better. I'm will try it and your method too

thanks

kungfoomasta

18-12-2008 22:53:12

Sorry for the insanely late response, but there isn't a visual studio solution for the project in svn?

If anybody has one and is willing to share it I'd be very thankful.

jacmoe

19-12-2008 03:07:20

Sorry for the insanely late response, but there isn't a visual studio solution for the project in svn?

If anybody has one and is willing to share it I'd be very thankful.
Cmake does.
I wonder what happens if you run it - I see a cmakelist.txt in the trunk.

CABAListic

19-12-2008 11:26:43

jacmoe's right - CMake is the one and only officially supported build system. It will generate the Visual Studio project files for you

kungfoomasta

20-12-2008 04:14:49

Ok so I found and installed CMake, I'm starting to read through the various documentation I can find, but can anybody break it down for a complete noob? (Obviously I'm not familiar with CMake)



C:\Users\Benjamin>cmake -G "Visual Studio 8 2005" D:\BuiltFromSource\ETL

-- Check for working CXX compiler: cl

-- Check for working CXX compiler: cl -- works

-- Detecting CXX compiler ABI info

-- Detecting CXX compiler ABI info - done

-- Could not find CEGUI, OIS or the Ogre CEGUI renderer. Not building sample.

CMake Error: The following variables are used in this project, but they are set

to NOTFOUND.

Please set them or make sure they are set and tested correctly in the CMake file

s:

OGRE_INCLUDE_DIR

   used as include directory in directory D:/BuiltFromSource/ETL

   used as include directory in directory D:/BuiltFromSource/ETL/src



-- Configuring incomplete, errors occurred!



C:\Users\Benjamin>

kungfoomasta

20-12-2008 04:33:27

ok I managed to figure things out using the cmake GUI. Such a pain to use, this tool..

KorgKeys

03-01-2009 09:53:38

Okay, I'm playing around with building a terrain editor using the latest svn version of ETL. Everything seems
to be working for the most part (deformation, splatting, painting, brush manipulation).

However, I'm running into a problem with splatting. I'm randomly getting the following error:

owed: OgreRenderQueueSortingGrouping.cpp:376: void Ogre::QueuedRenderableCollection::addRenderable(Ogre::Pass*, Ogre::Renderable*): Assertion `retPair.second && "Error inserting new pass entry into PassGroupRenderableMap"' failed.
(0) : fatal error C9999: *** exception during compilation ***

By random, I mean it sometimes happens as soon as I start splatting with a second texture, sometimes the third (or fourth or fifth ..). Sometimes it occurs as soon as I start splatting with the new texture, sometimes it occurs after I've been splatting with the new texture for some period of time. I can't reliably cause the error to occur.

Any thoughts?

CABAListic

03-01-2009 10:07:05

I haven't seen this, so I need you to find the exact place in the ETL code where it occurs. First of all, please update your Cg compiler to the latest version, because your version is one that crashes when errors in the application occur, and this Cg crash makes debugging impossible. Once you've upgraded Cg, you should be able to get a useful backtrace with a debugger. If you can tell me where the error originates from, I should be able to figure it out.

KorgKeys

03-01-2009 10:56:50

Okay, I updated Cg, compile my app with symbols and crashed it again.
The backtrace shows:



#0  0xb7f2d410 in __kernel_vsyscall ()

#1  0xb76bf085 in raise () from /lib/tls/i686/cmov/libc.so.6

#2  0xb76c0a01 in abort () from /lib/tls/i686/cmov/libc.so.6

#3  0xb76b810e in __assert_fail () from /lib/tls/i686/cmov/libc.so.6

#4  0xb7ccc908 in Ogre::QueuedRenderableCollection::addRenderable (this=0x9029270,

    pass=0x8e715a0, rend=0x8b45f9c) at OgreRenderQueueSortingGrouping.cpp:375

#5  0xb7cccb4f in Ogre::RenderPriorityGroup::addSolidRenderable (this=0x9029268,

    pTech=0x8b3be28, rend=0x8b45f9c, addToNoShadow=false)

    at OgreRenderQueueSortingGrouping.cpp:145

#6  0xb7cccbd1 in Ogre::RenderPriorityGroup::addRenderable (this=0x9029268,

    rend=0x8b45f9c, pTech=0x8b3be28) at OgreRenderQueueSortingGrouping.cpp:118

#7  0xb7cca54f in Ogre::RenderQueue::addRenderable (this=0x8781f68, pRend=0x8b45f9c,

    groupID=50 '2', priority=100)

    at ../../OgreMain/include/OgreRenderQueueSortingGrouping.h:528

#8  0xb7cca794 in Ogre::RenderQueue::addRenderable (this=0x8781f68, pRend=0x8b45f9c,

    groupID=<value optimized out>) at OgreRenderQueue.cpp:150

#9  0x08094116 in ET::Patch::_updateRenderQueue ()

#10 0xb52aa2c6 in Ogre::OctreeNode::_addToRenderQueue (this=0x8b15918, cam=0x8abb930,

    queue=0x8781f68, onlyShadowCasters=false, visibleBounds=0x8c2272c)

    at OgreOctreeNode.cpp:189

#11 0xb52a3543 in Ogre::OctreeSceneManager::walkOctree (this=0x85c0d38, camera=0x8abb930,

    queue=0x8781f68, octant=0x90290f8, visibleBounds=0x8c2272c, foundvisible=false,

    onlyShadowCasters=false) at OgreOctreeSceneManager.cpp:650

#12 0xb52a3606 in Ogre::OctreeSceneManager::walkOctree (this=0x85c0d38, camera=0x8abb930,

    queue=0x8781f68, octant=0x9029090, visibleBounds=0x8c2272c, foundvisible=false,

    onlyShadowCasters=<value optimized out>) at OgreOctreeSceneManager.cpp:668

#13 0xb52a3606 in Ogre::OctreeSceneManager::walkOctree (this=0x85c0d38, camera=0x8abb930,

    queue=0x8781f68, octant=0x9029028, visibleBounds=0x8c2272c, foundvisible=false,

    onlyShadowCasters=<value optimized out>) at OgreOctreeSceneManager.cpp:668

#14 0xb52a3606 in Ogre::OctreeSceneManager::walkOctree (this=0x85c0d38, camera=0x8abb930,

    queue=0x8781f68, octant=0x9028fc0, visibleBounds=0x8c2272c, foundvisible=false,

    onlyShadowCasters=<value optimized out>) at OgreOctreeSceneManager.cpp:668

#15 0xb52a3606 in Ogre::OctreeSceneManager::walkOctree (this=0x85c0d38, camera=0x8abb930,

    queue=0x8781f68, octant=0x9028f58, visibleBounds=0x8c2272c, foundvisible=false,

    onlyShadowCasters=<value optimized out>) at OgreOctreeSceneManager.cpp:668

#16 0xb52a3606 in Ogre::OctreeSceneManager::walkOctree (this=0x85c0d38, camera=0x8abb930,

    queue=0x8781f68, octant=0x9028ef0, visibleBounds=0x8c2272c, foundvisible=false,

    onlyShadowCasters=<value optimized out>) at OgreOctreeSceneManager.cpp:668

#17 0xb52a3606 in Ogre::OctreeSceneManager::walkOctree (this=0x85c0d38, camera=0x8abb930,

    queue=0x8781f68, octant=0x9028e88, visibleBounds=0x8c2272c, foundvisible=false,

    onlyShadowCasters=<value optimized out>) at OgreOctreeSceneManager.cpp:668

#18 0xb52a37bf in Ogre::OctreeSceneManager::walkOctree (this=0x85c0d38, camera=0x8abb930,

    queue=0x8781f68, octant=0x81617c8, visibleBounds=0x8c2272c, foundvisible=false,

    onlyShadowCasters=<value optimized out>) at OgreOctreeSceneManager.cpp:689

#19 0xb52a3948 in Ogre::OctreeSceneManager::_findVisibleObjects (this=0x85c0d38,

    cam=0x8abb930, visibleBounds=0x8c2272c, onlyShadowCasters=<value optimized out>)

    at OgreOctreeSceneManager.cpp:581

#20 0xb7d1ff4b in Ogre::SceneManager::_renderScene (this=0x85c0d38, camera=0x8abb930,

    vp=0x8a9f840, includeOverlays=true) at OgreSceneManager.cpp:1319

#21 0xb7b451be in Ogre::Camera::_renderScene (this=0x8abb930, vp=0x8a9f840,

    includeOverlays=<value optimized out>) at OgreCamera.cpp:403

#22 0xb7e16d10 in Ogre::Viewport::update (this=0x6) at OgreViewport.cpp:191

#23 0xb7ceeb1d in Ogre::RenderTarget::updateImpl (this=0x8416268)

    at OgreRenderTarget.cpp:113

#24 0xb7cee48c in Ogre::RenderTarget::update (this=0x8416268, swap=false)

    at OgreRenderTarget.cpp:501

#25 0xb7cd1461 in Ogre::RenderSystem::_updateAllRenderTargets (this=0x80ed070,

    swapBuffers=false) at OgreRenderSystem.cpp:112

#26 0xb7d0fd13 in Ogre::Root::_updateAllRenderTargets (this=0x80d6848) at OgreRoot.cpp:1132

#27 0xb7d0fe1a in Ogre::Root::renderOneFrame (this=0x80d6848) at OgreRoot.cpp:839

#28 0x08064bcc in OWED_Manager_App::processOneFrame (this=0x8b48270)

    at owed_manager_app.cpp:169

#29 0x0805a6ff in OWED_Main::main (this=0x80d66d8) at owed.cpp:132

#30 0x0805ac42 in main () at owed.cpp:65

(gdb)

CABAListic

03-01-2009 11:26:42

Hm, that's odd. There's only one ETL call in that trace, and that one is unlikely to have caused the error. So it must be something in the setup which fires off later on. Can you show me how you're setting up the terrain?

What about the ETL sample? Does it crash, too?

KorgKeys

03-01-2009 13:16:43

No, the ETSample program doesn't crash.

[ Actually I didn't realize that I *could* change splatting textures in ETSample. On my Linux box the drop down combo boxes for the images are all blank. However, I *can* click in the blank list area and select a new texture. ]

However, I think I finally found the source of the problem.

I found one reference to the error message in the Ogre forums:
http://www.ogre3d.org/forums/viewtopic.php?f=1&t=40839&start=0

Which links to the following bug:
http://www.ogre3d.org/mantis/view.php?id=130

As it turns out, I had also added a color selector to my edit terrain dialog box to make use of the paint support in ETL. I'm using MyGUI and I build all of my dialog boxes in code. In order to display the current color to the user I was using the MyGUI::RenderBox class to create a color swatch.

As it turns out, RenderBox creates it's own scene manager. Once I commented out the code for the RenderBox object the crashes went away. Of course after a test run with 30 textures, painting and deformations I was down to 14 FPS from a start of 200.

*sigh*

Glad I finally found the problem; sorry for having bothered you.

By the way; great plugin (ETL). Can't wait for the final version.

}:>

CABAListic

03-01-2009 15:22:34

bymulker

06-01-2009 09:04:09

hi,

i cant make cmake find OGRE_INCLUDE_DIR (CEGUI and OIS as well) although i set them to environment variables;
OGRE_INCLUDE_DIR = C:\ogrenew\OgreMain\include
CEGUI_INCLUDE_DIR = C:\ogrenew\Dependencies\include\cegui
OIS_INCLUDE_DIR = C:\ogrenew\Dependencies\include\ois

(Windows Vista, Visual C++ Express 2008)

can someone give a fast tip please?
i built etm without cmake but the sample application crashes.

CABAListic

06-01-2009 11:38:27

In the cmake GUI, switch advanced mode on and enter the missing variables yourself.

bymulker

06-01-2009 16:00:15

Stupid question! Thank you very much! works great

bymulker

07-01-2009 13:34:53

hi again,

i've made a quick copy paste of the sample code to my testbed. The terrain is ok but i can see neither splatting nor deformation;

Does anyone have any ideas?



void createTerrain(void)

	{

		size_t width = 257; 

		size_t height = 257;

		ET::RealArray2D heights(width, height, 0.5f);

		mTexture = "jungle_1.jpg";

		mRootNode = mSceneMgr->getRootSceneNode();

		mMaterial = new TerrainMaterial("TerrainMaterial", mRootNode);

		mBrush = new ET::ArrayBrush(ET::RealArray2D(2,2), Ogre::Vector2(10,10));



		mTerrain = new ET::FlatTerrain(heights, Ogre::Vector2(1, 1), 20);

		

		mColourMap = new ET::DynamicTexture("ETColourMap", "General", heights.sizeX()-1,

			heights.sizeY()-1, Ogre::PF_BYTE_RGB, Ogre::Vector2(1, 1));

		

		mLightMap = new ET::DynamicTexture("ETLightMap", "General", 128, 128, 

			  Ogre::PF_BYTE_RGB, Ogre::Vector2(1,1));

		

		mSplattingManager = new ET::SplattingManager("ETSplattingMap", "General",

			128, 128, Ogre::PF_BYTE_RGB, Ogre::Vector2(1,1), mTexture);

		

		ET::Page* page = new ET::Page("ETTerrain", mTerrain, 33, 33, true, 1);

		

		mMaterial->createMaterial();

		

		mSplattingManager->addListener(mMaterial->getSplattingMaterial());



		page->setMaterial(mMaterial->getMaterial());



		mRootNode->attachObject(page);



		//try some splat

		mTerrain->deform(50, 50, mBrush, 10);

		mTerrain->update();

                                

                               //and deform

		mSplattingManager->splat(50, 50, mBrush,"jungle_2.jpg", 10);

		mSplattingManager->getSplattingLayout()->update();

	



	}

KorgKeys

07-01-2009 14:08:00

No problem.
The white selection boxes are an error in the CEGUI skin, it's probably missing a font definition or something. But since I'm planning on ultimately replacing it with a custom skin, I haven't yet tried to correct the error.

I'm in the process of switching from MyGUI to CEGUI myself and I've run into the same problem with both combo and list boxes using the WindowsLook scheme. I came up with a workaround and tried it in your ETSample program. Just a couple of lines in GUIManager::initSelection():



for (Ogre::StringVector::const_iterator it = resList->begin(); it != resList->end(); ++it)

  {

    CEGUI::ListboxTextItem  *item = new CEGUI::ListboxTextItem(*it, n++);



    item->setSelectionBrushImage("WindowsLook", "Background");

    item->setTextColours(CEGUI::colour(0.0, 0.0, 0.0, 1.0));

    item->setSelectionColours(CEGUI::colour(0.7, 0.7, 1.0, 1.0));

    box->addItem(item);      

  }

Not the perfect solution but it does work.

Cheers!

stealth977

07-01-2009 18:53:29

Well i have a very strange problem, I am developing a scene builder application in which i decided to use ETL for terrain editing, i downloaded ETL, compiled it, but;

1 - The Debug build of ETL Sample Apllication can not display colourmap (It actually paints on it and i tried saving the colourmap before quit, and the saved colourmap is perfect) But, the colourmap is not bing rendered on the terrain!!

2 - The Release build works fine, no problems.

3 - For both of the builds, if i run them from Visual Studio 2008 editor, the colourmap doesnt work again (Again the colourmap gets modified but doesnt display)

4 - Also both builds cause a heap corruption error when AutoDelete is used. (Anyway if it is not used, lots of memory leaks)

I am using using Visual Studio 2008, Ogre 1.6, and ogre leak detection on in debug build. Ogre.log does not report any kind of errors.
Does anyone have any idea whats going wrong???
If you have precompiled libs, would you please send them to me so i can check if i am doing something wrong compiling the libs??
Thanks in advance =)

stealth977

10-01-2009 08:06:46

Ok First of all, I found the reason of Visual Studio going mad when deleting the TerrainManager object:

* Since your ETPatch class is a modified version of TerrainSceneManagers Patch class, it has the same problem with the Patch class; The Patch/ETPatch is already derived from MovableObject, but redefining the member attribute "mutable LightList mLightList" has issues in Visual Studio 2008 upon destruction of the object containing it. Since mLightList is already defined in MovableObject, there is no need to redefine it in ETPatch/Patch classes, removing the redefinition solves heap problems in VisualStudio 2008 (Since Patch Class lies in a seperate dll, i guess you wont get heap issues while using it, at least VisualStudio doesnt warn about it, but since your ETPatch is in a library and being linked to the application, and runs in applications own heap, visual studio goes crazy when its destructor is called)

I hope this helps people with similar problems.

Yet, I couldnt find the reason why colourmap is not being rendered on the mesh when running debug build or when running release build from Visual Studio IDE. If someone can pinpoint the issue please inform me

CABAListic

10-01-2009 12:31:50

Hm, interesting, so maybe this is an ancient bug in the TerrainSceneManager I "inherited". Anyway, thanks, I'll look into it.

As for the colour map, this sounds like some shader issue. What graphics card do you have? Do you have any errors in the Ogre.log? Does it help changing the rendersystem?

BTW: I read your posts in the Ogre showcase forum. I agree that Geomipmapping is not in itself particularly suitable for large terrains, and if you need those, my library will probably not work for you

It was designed primarily for my own need, which is relatively small maps for a strategy type game. Of course, Geomipmapping can be used in large worlds via paging, assuming that you just want a large walkable terrain like in a RPG type game. But it sounded more like you want to display a huge terrain all at once (kind of like in a flight sim or for editing purposes) in which case you probably do need a different rendering approach. You can optimise ETM/ETL for the purpose slightly by choosing a bigger patch size (reduces batches; the patch size in the demo is very small and better for culling, you would probably want a patch size of at least 513 or even 1025) and using higher LOD levels, but there will still be a limit, I imagine.
Anyway, I don't know much about clipmaps. I did research into alternatives to geomipmapping at one point, but clipmaps are, as far as I know, patented, so I didn't bother. The rest of the alternatives are mostly just variations on geomipmapping.

stealth977

10-01-2009 18:13:49

Cabalistic:

Thank you for your interest

I finally solved the second problem, Actually having different results in debug and release builds was confusing a lot. I changed my cg.dll to 2.1, downloaded a fresh Ogre v1.6 from svn, rebuilt everything and at last i had a stable result from both builds: both release and debug builds were not showing the colourmap =)

But then i realised, it may be because of depth fighting, and after checking shadergenerator and terrainmaterial files i found out that in TerrainMaterial.cpp it was using VSLodMorph2 as vertex shader, but your shader generator generates a non-morphing shader, so somehow the coordinates each create are somewhat different (Floating point issues?) , changing the VSLodMorph2 to a new non-morphing vshader for the colourmap and lightmap passes solved the problem =)

I hope this helps people having similar problems (if any?

)

Also, I must admit that your work here (ETM/ETL) is really great. It makes terrain editing very easy and its being modular makes it easy to integrate in any editor. My only problem is the geomipmapping aproach being less memory/batch friendly.
Maybe increasing patch sizes would help a bit.

Anyway i decided to include your ETL in my editor. I dont have enough time and motivation to recreate all those pretty terrain editing options ETL already has, for my clipmap renderer. Spending time on it damages my motivation to develop the scene builder =) And i really want to concentrate on my scene builder application.

Thanks for your efforts on ETM/ETL, because it helps me concentrate on my project and i am sure it helps other people too in their projects.

CABAListic

10-01-2009 18:46:31

Ah yes, you're right, the materials and shaders are not completed yet, I forgot that

And do try larger patch sizes, it will significantly reduce batching (at the cost of culling, but for your needs it seems that culling wouldn't be too efficient, anyway).

stealth977

10-01-2009 20:53:53

Well, actually i do not need anything, i am not on a project either. Its just a pleasure to build my application and if i can succeed in completing what i plan for this editor, then it will be a nice contribution to the community.
So, about the terrain sizes: Its just that i want every user to have the opportunity to edit anything they want, so i do not want to say: your terrain editing opportunity is limited to AAAxBBB size maps if you want to use this editor.

But, i think i have a solution to this problem, if user wants to edit lets say a 4096x4096 map, he/she can create 16 instances of ETL with 1024x1024 maps, or maybe 4 instances of 2048x2048 maps in the editor (depending on their computers' power of course), and can make only the needed ones visible at a time, so the frame rate will be affordable.

stealth977

12-01-2009 10:28:32

  void EditableObject::determineEditRect(Real x, Real z, const Brush* brush)

  {

    Rectangle bounds = brush->getBoundingRect();

    bounds.left += x;

    bounds.right += x;

    bounds.top += z;

    bounds.bottom += z;

    bounds.left = ceil(bounds.left / mGridScale.x);

    bounds.right = floor(bounds.right / mGridScale.x);

    bounds.top = ceil(bounds.top / mGridScale.y);

    bounds.bottom = floor(bounds.bottom / mGridScale.y);

    // get the according integer coordinates

    mEditedRect.x1 = (unsigned int) max<Real>(0, bounds.left);

    mEditedRect.y1 = (unsigned int) max<Real>(0, bounds.top);

    mEditedRect.x2 = (unsigned int) min<Real>(mGridScale.x*(mGridWidth-1), bounds.right);

    mEditedRect.y2 = (unsigned int) min<Real>(mGridScale.y*(mGridHeight-1), bounds.bottom);

  }

I found a little bug here =)

mEditedRect.x2 = (unsigned int) min<Real>(mGridScale.x*(mGridWidth-1), bounds.right);
mEditedRect.y2 = (unsigned int) min<Real>(mGridScale.y*(mGridHeight-1), bounds.bottom);

Here, we are already in Grid Space, so multiplication causes the right and bottom bounds to go out of grid space, causing memory owerwrite somewhere it shouldnt ----> Crash =)

Btw, i see its still revision 39, are you planning an update soon?

CABAListic

12-01-2009 13:55:43

I do plan to, but I can't promise because my studies currently cut heavily into my amount of free time...

ppClarity

13-01-2009 15:23:13

CABAListic

22-01-2009 00:57:20

But, i think i have a solution to this problem, if user wants to edit lets say a 4096x4096 map, he/she can create 16 instances of ETL with 1024x1024 maps, or maybe 4 instances of 2048x2048 maps in the editor (depending on their computers' power of course), and can make only the needed ones visible at a time, so the frame rate will be affordable.
I've found that 1024x1024 areas tend to lag quite a bit on my AMD 4600+ x2, 2Gb RAM, 512Mb Nvidia 8800GT. I've been debating testing 256^2 or even 128^2 chunks next chance I get to play with my code.

Just did a quick test. I created a 1025x1025 map and used 33x33 patches, as the ETL sample currently does. The result is very laggy, framerate at about 20 fps, not very convincing. Now I changed the patch size to 257x257, and the overall framerate went up to the range of 300 fps (!). So the patch size has a vital role for performance, because it effectively controls the batch count.

I learnt two less fortunate things from this experiment:
1) There's a rendering bug, or more likely even a setup bug, that doesn't show with the 33x33 patches. There are visible gaps between the patches, and at certain edit points it would actually raise whole patches instead of the selected target area. I need to track this down.
2) Terrain editing gets very, very slow. Unfortunately updating has to be done on a per patch basis, so the larger the patch, the more time the update needs. But I'm not quite content with how slow it is. I will investigate potential optimisations.
Updating a patch consists of two phases: First we need to update the vertex buffers. Here only vertices are updated which were changed, you can't really make this any more effective than it is. But the second phase investigates all the vertices to determine the camera distances at which to switch LOD levels. It would be nice to also localise this update to the changed parts, but that is not as easy (if at all doable). So I'm considering using a less expensive (and less accurate) calculation to get the camera switch distances and also considering the possibilities of gradual updates, i. e. split the update into several calls.

stealth977

22-01-2009 15:34:43

CABAListic:

I have a few questions;

1 - Is there an official way to restore SplatMaps (Ex: While loading terrain in an editor) Currently i hacked your code to do it =)(copied a public version of addtexture with a TextureList instead of single texture also a bit more changes too, I pass the splat textures used to that function which in turn creates the layers, then i get SplatMaps and update their data with the saved maps)



  void SplattingManager::setTextures(const TextureList& textures)

  {

    mTextures = textures;

    mNumUsedChannels = 0;

    for(unsigned int i = 0;i < mTextures.size();i++)

    {

	   mTexChanMap[mTextures[i]] = mNumUsedChannels;

       mDirtyRects.push_back(Rect(0,0,mGridWidth - 1, mGridHeight - 1));

	   ++mNumUsedChannels;

    }

    mSplattingLayout->setNumChannels(mNumUsedChannels);



    // inform listeners about texture list change

    for (vector<SplattingListener*>::iterator l = mListeners.begin(); l != mListeners.end(); ++l)

    {

      (*l)->notifySplattingTextures(mTextures);

      (*l)->notifySplattingMaps(mSplattingLayout->getSplattingMapNames());

    }

  }

2 - I didnt realise any gaps in the terrain yet, at what configuration does it show?

3 - Do you know any affordable way to implement UNDO on terrain deform/splat/paint, using negative direction wouldnt work you know...

Thanx, Stealth

CABAListic

22-01-2009 16:09:32

1) Is in the works.

2) Begins showing (for me) at a patch size of 257x257. 129x129 seems to be fine, as is anything below. I haven't yet figured out what goes wrong.

3) Is also in the works (or rather, planned). Some pages back in this thread I described roughly what I have in mind. Basically I'm going to provide a sort of History manager which can take diffs of modified EditableObjects and can restore them to a previous state.

CABAListic

23-01-2009 15:13:47

Ok, I fixed the rendering bug with larger patch sizes. It was, of course, simply caused by an overflow. I had without much thought created the index buffers with 16 bit precision. This works fine up to 129x129 patches, but 257x257 = 66049 is too large for 16 bit. I templatised the creation code, it now switches to 32 bits when needed.
This bug affects ETM, too. Seeing as the map sizes I'm personally targeting in my project are in the size of 257x257, I never had any need for patches of that size and never tested that :roll:

CABAListic

27-01-2009 20:50:43

Ok, I implemented caching for the LOD recalculations, so that editing 257x257 patches is now possible. Patches of size 513x513 is still borderline, but then again, 513x513 is borderline for the geomipmapping approach, anyway.

I'm now going to finally get to proper save/load functionalities. After that I'll implement the history manager, and as a final step I'll investigate gradual construction of terrain pages/patches so that you can stream them in a paging solution or display a progress bar.

kungfoomasta

27-01-2009 21:27:40

Awesome! I was just about to ask whats left on your TODO list, but I think you summed it up.

SongOfTheWeave

02-02-2009 07:26:22

CABAListic

02-02-2009 11:06:08

No, I want to have ETL feature-complete, then I'll do an official beta release to hunt all those nasty little bugs which keep crawling into software

This will almost certainly happen in March, this month I'm busy with exams again. I may or may not finish save/load functionality before, but no promises.

rubasurex

09-02-2009 15:27:40

There appears to be a problem with the FlatTerrain::getHeightAt(Real x, Real z) method. It does not appear to be returning accurate results, unlike ETM 2.3.1, which returns very accurate heights. Unless of course I've done something wrong. To correct the issue for my project I have replaced the getHeightAt method in ETL with the ETM version (with a couple of modifications) and it appears to work well.

Here is a comparison screen shot to demonstrate the problem. The original ETL getHeightAt method is shown in the shot on the left. The modified ETM getHeightAt method running in ETL is shown on the right.

Here is the code I dropped in from ETM.

    // scale down

    x /= mVertexScale.x;

    z /= mVertexScale.z;

    if (x < 0.0 || x > mSizeX-1 || z < 0.0 || z > mSizeZ-1)

    {

      // out of bounds

      return 0.0f;

    }



    // retrieve height from heightmap via bilinear interpolation

    size_t xi = (size_t) x, zi = (size_t) z;

    float xpct = x - xi, zpct = z - zi;

    if (xi == mSizeX-1)

    {

      // one too far

      --xi;

      xpct = 1.0f;

    }

    if (zi == mSizeZ-1)

    {

      --zi;

      zpct = 1.0f;

    }



    // retrieve heights

    float heights[4];

    heights[0] = mHeights.at(xi, zi);

    heights[1] = mHeights.at(xi, zi+1);

    heights[2] = mHeights.at(xi+1, zi);

    heights[3] = mHeights.at(xi+1, zi+1);



    // interpolate

    float w[4];

    w[0] = (1.0 - xpct) * (1.0 - zpct);

    w[1] = (1.0 - xpct) * zpct;

    w[2] = xpct * (1.0 - zpct);

    w[3] = xpct * zpct;

    float ipHeight = w[0]*heights[0] + w[1]*heights[1] + w[2]*heights[2] + w[3]*heights[3];



    // scale to actual height

    ipHeight *= mVertexScale.y;



    return ipHeight;

I'm assuming there is particular reason why this method was changed in ETL and perhaps what I've done will have other side effects (I don't know enough about ETL yet to tell). But so far for my project, everything seems to be working fine.

On a side note, ETL appears to take a very long time to build its pages. I was originally using ETM and it would load my terrain, complete with splatting and lightmap in about 5 seconds. ETL on the other hand, to load the same heightmap, but without splatting or a lightmap takes about twice as long. Is there any plan to look into improving the load times in ETL?

CABAListic

09-02-2009 15:47:19

The reason I changed the getHeightAt function is that the ETM implementation gives interpolated results which are not triangle-accurate. The new implementation is supposed to be just that, but it may of course contain bugs. In any case, the interpolated version will be made available under the name interpolateHeightAt, because it gives "smoother" results which are nice for some applications.

As for loading speed: The way things are loaded is not final. I'm quite certain I know why it's loading slower, and yes, that will be fixed. In March

rubasurex

09-02-2009 23:29:54

OK, no problem. Sounds good.

I'll just stick with my little hack for now until its ironed out.

rubasurex

11-02-2009 13:58:14

Have a quick question regarding loading splatting maps and textures. I've upgraded from ETM and I'm trying to load in the coverage maps and textures that I was using into the splatting material, but it doesn't appear to be working for me. I'm guessing that either I'm doing something wrong or the coverage maps generated by ETM are not compatible?

Here is the code I'm using. All I see is the base texture.

material = new TerrainMaterial("ETMaterial", sm->getRootSceneNode());



ET::TextureList maps;

maps.push_back("c.0.png");

maps.push_back("c.1.png");

material->setSplattingMaps(maps);



ET::TextureList textures;

textures.push_back( "s.0.png" );

textures.push_back( "s.1.png" );

textures.push_back( "s.2.png" );

textures.push_back( "s.3.png" );

textures.push_back( "s.4.png" );

textures.push_back( "s.5.png" );

material->setSplattingTextures(textures);



material->createMaterial();

On a side note. Is it possible to save out the maps and textures from the splatting material at runtime?

stealth977

11-02-2009 18:00:51

It is a bit different in ETL, currently there is no official loading interface, with advice from CABAListic that is how i implemented loading code in Ogitor.



void CETLEditor::LoadTerrain(Ogre::String FilesDir,int size, float scaleX, float scaleY, int SplatSize, int LightMapSize, int ColourMapSize, int CellSpan, ET::TextureList splats )

{

  m_MapSize = size;

  m_MapScale = Ogre::Vector3(scaleX,scaleY,scaleX);

  m_SplatMapSize = SplatSize;

  m_LightMapSize = LightMapSize;

  m_ColourMapSize = ColourMapSize;

  if(CellSpan > 0) m_CellSpan = CellSpan;



  Ogre::ResourceGroupManager::getSingleton().addResourceLocation(FilesDir, "FileSystem", "ETLFiles");

  Ogre::String fName = GetName() + ".";



  Ogre::Image heightmap_image;

  heightmap_image.load(fName + "heightmap.png","ETLFiles");  



  Ogre::Image colourmap_image;

  colourmap_image.load(fName + "colourmap.png","ETLFiles");



  Ogre::Image lightmap_image;

  lightmap_image.load(fName + "lightmap.png","ETLFiles");



  if (heightmap_image.getData() != NULL && colourmap_image.getData() != NULL && lightmap_image.getData() != NULL)

  {

    ET::RealArray2D heights = ET::loadRealArrayFromImage(heightmap_image);

    ET::ColourArray2D colourMap = ET::loadColourArrayFromImage(colourmap_image);

    ET::ColourArray2D lightMap = ET::loadColourArrayFromImage(lightmap_image);



    m_Terrain = new ET::FlatTerrain(heights, Ogre::Vector2(m_MapScale.x, m_MapScale.x), m_MapScale.y);

//    m_Handle = new ET::Page("ETL_Terrain", m_Terrain, 65, 65, true, 1);

    m_Handle = new ET::Page("ETL_Terrain", m_Terrain, 129, 129, true, 4, 0.05f, ET::VO_NORMALS | ET::VO_LODMORPH | ET::VO_TANGENTS, 0.01f);

  

	float scaleSplatMap = m_MapScale.x * (float)m_MapSize / (float)m_SplatMapSize;

    m_SplattingManager = new ET::SplattingManager("ETSplattingMap", "General", m_SplatMapSize, m_SplatMapSize,

	  Ogre::PF_BYTE_RGB, Ogre::Vector2(scaleSplatMap, scaleSplatMap), splats[0]);

	

    setColourMap(m_ColourMapSize, m_ColourMapSize, colourMap);

    setLightMap(m_LightMapSize, m_LightMapSize, lightMap);

    float fRepeat = (float)(m_MapSize - 1) / (float)m_CellSpan;

    

	if(splats.size() > 0 && !splats[0].empty()) 

	 m_Texture = splats[0];

	else

	 splats[0] = m_Texture;



	m_Material->setTextureRepeat(fRepeat,fRepeat);

	m_Material->createMaterial();

    m_SplattingManager->addListener(m_Material->getSplattingMaterial());

	m_SplattingManager->addListener(this);

    m_Handle->setMaterial(m_Material->getMaterial());

    

    // register terrain objects to be automatically updated

    m_Updater->addUpdatable(m_Terrain);

    m_Updater->addUpdatable(m_SplattingManager);

    m_Updater->addUpdatable(m_ColourMap);



	m_Brush->setSize(Ogre::Vector2(10000,10000));

	for(unsigned int i = 1;i < splats.size();i++)

	{

     if(splats[i].empty()) 

	    splats[i] = splats[i - 1];	

	  m_SplattingManager->splat(0,0,m_Brush,splats[i],0.2f); 

	}

	m_Brush->setSize(Ogre::Vector2(12,12));



	int count = (splats.size() + 2) / 3;

    for(int i = 0;i < count;i++)

	{

	  Ogre::String mapname = fName + "coverage" + Ogre::StringConverter::toString(i) + ".png";

	  Ogre::Image covmap;

	  covmap.load(mapname,"ETLFiles");

	  ET::ColourArray2D covArray = ET::loadColourArrayFromImage(covmap);

	  m_SplattingManager->getSplattingLayout()->getSplattingMap(i)->setColourArray(covArray);

	}



	m_RootNode->attachObject(m_Handle);

  }

}

I hope it helps

rubasurex

12-02-2009 00:45:59

Hey thanks stealth, very much appreciated.

BTW, great work your doing there with Ogitor!

Bob

16-03-2009 16:08:34

hi,
I tried to compile the etl version from the svn trunk.

I'm using eclispe + cdt plugin + mingw.

But I get the following compile error:



In file included from C:\etl\EditableTerrain\src\ETPage.cpp:26:

C:/etl/EditableTerrain/include/ETTerrainDescription.h:41: error: base `ET::Editable' with only non-default constructor in class without a constructor

C:\etl\EditableTerrain\src\ETPage.cpp: In member function `void ET::Page::createPatches(const Ogre::String&, unsigned int, Ogre::Real, unsigned int, Ogre::Real)':

C:\etl\EditableTerrain\src\ETPage.cpp:114: error: cannot allocate an object of type `ET::Patch'

C:\etl\EditableTerrain\src\ETPage.cpp:114: error:   because the following virtual functions are abstract:

C:/etl/EditableTerrain/include/ETUpdatable.h:43: error:  virtual void ET::Updatable::update()

C:\etl\EditableTerrain\src\ETPage.cpp: At global scope:

C:\etl\EditableTerrain\src\ETPage.cpp:153: error: prototype for `void ET::Page::notifyDestroyed()' does not match any in class `ET::Page'

C:/etl/EditableTerrain/include/ETPage.h:86: error: candidate is: virtual void ET::Page::notifyDestroyed(ET::Editable*)

mingw32-make[2]: Leaving directory `C:/etl/bin'

mingw32-make[2]: *** [EditableTerrain/CMakeFiles/EditableTerrain.dir/src/ETPage.cpp.obj] Error 1

mingw32-make[1]: Leaving directory `C:/etl/bin'

mingw32-make[1]: *** [EditableTerrain/CMakeFiles/EditableTerrain.dir/all] Error 2

mingw32-make: *** [all] Error 2

CABAListic

16-03-2009 21:11:40

Current SVN version does not compile, sorry. I had to switch computers, so I had to check in some development work which is not yet complete. You need to go back to revision 51 or 50, check the log, the first broken revision is clearly marked as such.

kungfoomasta

25-03-2009 17:10:07

Hey CABAListic! I was just curious to know if there have been any recent developments on ETL v3?

CABAListic

25-03-2009 18:16:20

I'm on it! I just underestimated the scope of the latest changes I began to clean up the interface. But at least in the process I'll be adding the necessary functionality to load an ETL component piece by piece instead of as a single, whole step, which is vital for loading progress display (or paging, even though hopefully with sinbad's ongoing work the need for a paged ETL will be zero). Bear with me

mkiv

25-03-2009 18:48:04

Is it currently possible to draw a grid overlay onto the terrain by using one of the splatting textures?
Would I need to create a custom material script - or is this something planned for etmv3 that I should just wait for?

Cheers

CABAListic

25-03-2009 19:07:50

No grid planned - unless a wireframe rendering of the terrain is enough for you, this is nothing you can really do with textures. Resolution won't be good enough. Try a ManualObject to construct a grid on top of the terrain.

florinudrea

06-04-2009 15:07:13

Is there a way to create the terrain into another position than the default. For now the top left corner is created into 0,0,0 but i would like to have my terrain centered into 0,0,0. I suppose somehow i need to generate the vertices relative to 0, 0, 0 or should i move the parent node of the terrain to center it?

CABAListic

06-04-2009 15:09:17

Best use a MovableContainer, attach the FlatTerrain to it, then attach the Container and the Page to a SceneNode und move that wherever you wish.

rubasurex

13-04-2009 09:32:36

stealth977: I've noticed what appears to be a problem with the loading code you use in Ogitor. Perhaps CABAListic can comment on this. Take a look at the following snippet from your previously listed LoadTerrain method.



m_Brush->setSize(Ogre::Vector2(10000,10000));

for(unsigned int i = 1;i < splats.size();i++)

{

   if(splats[i].empty()) 

      splats[i] = splats[i - 1];	

   m_SplattingManager->splat(0,0,m_Brush,splats[i],0.2f); 

}

m_Brush->setSize(Ogre::Vector2(12,12));



int count = (splats.size() + 2) / 3;

for(int i = 0;i < count;i++)

{

   Ogre::String mapname = fName + "coverage" + Ogre::StringConverter::toString(i) + ".png";

   Ogre::Image covmap;

   covmap.load(mapname,"ETLFiles");

   ET::ColourArray2D covArray = ET::loadColourArrayFromImage(covmap);

   m_SplattingManager->getSplattingLayout()->getSplattingMap(i)->setColourArray(covArray);

}

The problem is that after loading the terrain, every splat texture ends up with a dirty rect the full size of the terrain. Every time splat is called, the splatting manager updates a dirty rect that is supposed to be the smallest possible rect that encapsulates a texture's influence on the terrain (CABAListic correct me if I'm wrong). By setting the brush to cover the whole terrain, splatting the terrain with every texture and then loading the coverage maps, you always end up with the dirty rects set to the entire terrain surface size.

I believe this will add a performance hit when splatting as the splatting manger searches the dirty rect to ensure the texture is being used. This means that the splatting manager will always search the entire coverage map, every time you splat.

To solve the problem I've added two new methods to ETL. The first exposes the vector of dirty rects, which I then iterate through and save out to file when saving the map. The second method allows you to set the dirty rect for a given texture index. That way you can read the dirty rects back from disk when loading the map and then set them into the splatting manager.