How do I create a 3D input window?

Hello you all,

I'd like to use MyGUI in 3D space, so not only windows as overlay, but as 3D displays & controls.

Here is a very good youtube example: https://www.youtube.com/watch?v=e78Hfo5rIyU

Any clues how that can be done?

Your help ist very much appreciated!

Kind regards,
Corak

Graphic Example:

You need to render MyGUI into texture and then use this texture on plane or any 3d object. Then with raytracing get UV coordinate of intersection and use it as mouse position.

There is UnitTest_RttLayer (UnitTexts seciton is disabled in CMake by default), that implement such logic: https://www.youtube.com/watch?v=J-FeieAFLGM

Dear Altren,

thank you for your reply! Thats exactly what I am looking for.

Do you have a code snippet or example how the render a MyGUI-Window to a texture by chance?

Kind regards from Germany,
Corak

Hello Altren,

I tried to use the sample, but it doesn't compile.

What I've done:
I copied MyGUI_RTTLayer.cpp and MyGUI_RTTLayer.h from here into my "MyGUI\include\MYGUI" - Folder.

In my code I call:
#include "MyGUI_RTTLayer.h"

/********/

MyGUI::FactoryManager::getInstance().registerFactory<MyGUI::RTTLayer>("Layer");
// This line above crashes the compiling

MyGUI::ResourceManager::getInstance().load("Layers.xml");
MyGUI::LayoutManager::getInstance().loadLayout("Layers.layout");

My IDE (MSVS2010) throws the error:
Error 2 error LNK2001: Nicht aufgelöstes externes Symbol ""public: __thiscall MyGUI::RTTLayer::RTTLayer(void)" (??0RTTLayer@MyGUI@@QAE@XZ)". D:\#\OgreProject\OgreProject\Main.obj1
in English something like "non dissolved external symbol"

PS: I am using the Ogre-Addon-Collection form Transporter (source)

You need to add MyGUI_RTTLayer.cpp file into your build, rather than just copy it into include folder.

Also use sources from MyGUI repository instead of outdated (and my be modified) copy.

Thank you well, Altren! It works

The include into the project did it and now its looking quite impressive

Then with raytracing get UV coordinate of intersection and use it as mouse position.
About raytracing: Are there any libaries/funktions/widgets in MyGUI yet included that do the job (getting mouse position on texture and compare it with de defined widgets)? Or do I have to implement this from scratch?

Kind regards,
- Corak

This logic is implemented im UnitTest_Layers in two classes: SceneObject and RTTLayer.

Dear Altren,
I find it extremely difficult to crawl through the demo files, sorry for that . I just can't find how the logic is to be used. Right now I found the files, included SceneObject.cpp into my project (as I've done with MyGUI_RTTLayer.cpp) and included SceneObject.h into my header.

MyGUI::InputManager::getInstance().injectMouseMove(arg.state.X.abs, arg.state.Y.abs, arg.state.Z.abs);This code works for the normal 2D-layer (the hover effects are working, but not on the RTTLayer-Texture).

How do I invoke the SceneObject? Which function is used to retrieve the information, if a button on the object with the rendered Layer is clicked or hovered?

Kind regards,
- Corak

Over the last weekend I tried to get the UnitTest_Layers working, but without any success. Than I analyzed the functions called one by one.

In UnitTest_Layers/DemoKeeper.h i find 3 virtual void functions looking very promising:
virtual void injectMouseMove(int _absx, int _absy, int _absz);
virtual void injectMousePress(int _absx, int _absy, MyGUI::MouseButton _id);
virtual void injectMouseRelease(int _absx, int _absy, MyGUI::MouseButton _id);
In ScreneObject.h i find those promising private functions:
bool isIntersectMesh(int& _x, int& _y, const Ogre::Ray& _ray, int _texture_width, int _texture_height) const;
Ogre::Vector2 getCoordByTriangle(Ogre::Vector3 _position, const Ogre::Vector3& _corner0, const Ogre::Vector3& _corner1, const Ogre::Vector3& _corner2) const;
Ogre::Vector2 getCoordByRel(Ogre::Vector2 _position, const Ogre::Vector2& _corner0, const Ogre::Vector2& _corner1, const Ogre::Vector2& _corner2) const;

void updateData();In MyGUI_RTTLayer.h the ScreneObject seems to be constructed without further work needed, as RTTLayer gets SceneObject as parent class:
class RTTLayer :
public OverlappedLayer
#ifdef MYGUI_OGRE_PLATFORM
, public demo::SceneObject
#endif
{
MYGUI_RTTI_DERIVED( RTTLayer )

My apoligies - can you please a bit more precise, how the intersection-calculation is to include in the main program? Or does anyone have a further hint, how to progress?

Kind regards -
Corak

Register your new layer class: std::string layerCategory = MyGUI::LayerManager::getInstance().getCategoryName();
MyGUI::FactoryManager::getInstance().registerFactory<MyGUI::RTTLayer>(layerCategory);Load file such as Media\UnitTests\UnitTest_Layers\Layers.xml with a custom RTTLayers added in it. MyGUI::ResourceManager::getInstance().load("Layers.xml");
Create entity/entities with name used in Layers.xml and attach it to scene node as usual.
In layouts and widgets that are supposed to be displayed on your mesh use layers names from Layers.xml.

Dear Altren,

the widgets are displayed correctly. But you still can't click on them and use their delegated functions, so there is no mouse input at all.

The global variable is Ogre::Entity* entGround;
The layers init looks like std::string layerCategory = MyGUI::LayerManager::getInstance().getCategoryName();
MyGUI::FactoryManager::getInstance().registerFactory<MyGUI::RTTLayer>(layerCategory);

MyGUI::ResourceManager::getInstance().load("Layers.xml");
MyGUI::LayoutManager::getInstance().loadLayout("Layers.layout");
The button callback is (coded works on any overhead-layer) button001 = mGUI->findWidget<MyGUI::Button>("GMyFirstButton");
button001->eventMouseButtonClick += MyGUI::newDelegate(&Call_my_event);
This is the mesh Ogre::Plane plane(Ogre::Vector3::UNIT_Y, 0);
Ogre::MeshManager::getSingleton().createPlane("ground", Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME,
plane, 4, 4, 20, 20, true, 1, 1, 1, Ogre::Vector3::UNIT_Z);
Ogre::Entity* entGround = mSceneMgr->createEntity("GroundEntity", "ground");
mSceneMgr->getRootSceneNode()->createChildSceneNode()->attachObject(entGround);And here is MyGUI casting the window onto the texture "RTTTexture1" as defined in Layers.xml (<Property key="TextureName" value="RTTTexture1"/>), see below Ogre::MaterialPtr materialx = Ogre::MaterialManager::getSingleton().create("BrowserMaterial",Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);
Ogre::Technique *technique = materialx->createTechnique();
technique->createPass();
materialx->getTechnique(0)->getPass(0)->setLightingEnabled(false);
materialx->getTechnique(0)->getPass(0)->setDepthWriteEnabled(false);
materialx->getTechnique(0)->getPass(0)->createTextureUnitState( "RTTTexture1");
entGround->setMaterialName("BrowserMaterial");
Layers.layout defines the RTTLayer contents as follows, using layer="RTTLayer1"<?xml version="1.0" encoding="UTF-8"?>
<MyGUI type="Layout" version="3.2.0">
<Widget type="Window" skin="WindowC" position="0 0 800 800" layer="RTTLayer1" name="_Main">
<Property key="Caption" value="Layered to texture..."/>
<Widget type="Button" skin="Button" position="25 20 200 20" align="HStretch Top" name="GMyFirstButton">
<Property key="Caption" value="Klick here"/>
</Widget>
</Widget>
<CodeGeneratorSettings/>
</MyGUI>
In Layers.xml "RTTLayer1" is defined as type="RTTLayer":<?xml version="1.0" encoding="UTF-8"?>
<MyGUI type="Layer" version="1.0">

<Layer name="Back" overlapped="false" pick="true"/>
<Layer name="Overlapped" overlapped="true" pick="true"/>
<Layer name="Middle" overlapped="false" pick="true"/>
<Layer name="Modal" overlapped="true" pick="true"/>
<Layer name="Main" overlapped="false" pick="true"/>

<Layer type="RTTLayer" name="RTTLayer1" pick="true">
<Property key="TextureSize" value="800 800"/>
<Property key="TextureName" value="RTTTexture1"/>
<Property key="SceneManager" value="BaseSceneManager"/>
<Property key="Camera" value="BaseCamera"/>
<Property key="Entity" value="GroundEntity"/>
<Property key="Material" value="BrowserMaterial"/>
</Layer>

<Layer name="Popup" overlapped="true" pick="true"/>
<Layer name="FadeMiddle" overlapped="false" pick="false"/>
<Layer name="Info" overlapped="true" pick="true"/>
<Layer name="ToolTip" overlapped="false" pick="false"/>
<Layer name="DragAndDrop" overlapped="false" pick="false"/>
<Layer name="FadeBusy" overlapped="false" pick="false"/>
<Layer name="Pointer" overlapped="false" pick="false"/>
<Layer name="Fade" overlapped="false" pick="false"/>
<Layer name="Statistic" overlapped="false" pick="false"/>

</MyGUI>
MyGUI_RTTLayer.cpp and SceneObject.cpp are included in the project and the Included headers are:#include "MyGUI.h"
#include "MyGUI_OgrePlatform.h"
#include "MyGUI_RTTLayer.h"
#include "SceneObject.h"
#include "MyGUI_InputManager.h"

The window is rendered correctly on the mesh, even if I call functions like button001->setcaption("xyz") the texture gets updates. The only thing that doesn't work yet is the mouse input.

Which part have I missed - or do I have to define anything else?

Kind regards,
- Corak

I guess I wrote wrong order of creation: entity must be created before loading layers file, because when RTTLayer is initialized it finds related entity.

Hmm, I've changed the order but with no effect.

As the texture is rendered correctly and only the mouse input doesn't work - is there anything I have to consider/implement mouse-input-related?

Maybe there is an error in the included files. Here are my versions:

MyGUI_RTTLayer.h
/*!
@file
@author Albert Semenov
@date 12/2009
*/

#ifndef __MYGUI_RTT_LAYER_H__
#define __MYGUI_RTT_LAYER_H__

#include "MyGUI_Prerequest.h"
#include "MyGUI_Types.h"
#include "MyGUI_OverlappedLayer.h"
#include "SceneObject.h"

namespace MyGUI
{

class RTTLayer :
public OverlappedLayer
//#ifdef MYGUI_OGRE_PLATFORM
, public SceneObject
// , public demo::SceneObject
//#endif
{
MYGUI_RTTI_DERIVED( RTTLayer )

public:
RTTLayer();
virtual ~RTTLayer();

virtual void deserialization(xml::ElementPtr _node, Version _version);
virtual void renderToTarget(IRenderTarget* _target, bool _update);

virtual ILayerItem* getLayerItemByPoint(int _left, int _top) const;
virtual IntPoint getPosition(int _left, int _top) const;

void setTextureSize(const IntSize& _size);
void setTextureName(const std::string& _name);

private:
MyGUI::ITexture* mTexture;
mutable IntPoint mOldPoint;
IntSize mTextureSize;
std::string mTextureName;
bool mOutOfDate;
};

} // namespace MyGUI

#endif // __MYGUI_RTT_LAYER_H__
MyGUI_RTTLayer.cpp
/*!
@file
@author Albert Semenov
@date 12/2009
*/

//#include "Precompiled.h"
#include "MyGUI_LayerItem.h"
#include "MyGUI_RTTLayer.h"
#include "MyGUI_Enumerator.h"
#include "MyGUI_FactoryManager.h"
#include "MyGUI_RenderManager.h"
#include "MyGUI_Gui.h"
#include "MyGUI_LayerNode.h"

namespace MyGUI
{

RTTLayer::RTTLayer() :
mTexture(nullptr),
mOutOfDate(false)
{
}

RTTLayer::~RTTLayer()
{
if (mTexture)
{
MyGUI::RenderManager::getInstance().destroyTexture(mTexture);
mTexture = nullptr;
}
}

void RTTLayer::deserialization(xml::ElementPtr _node, Version _version)
{
Base::deserialization(_node, _version);

MyGUI::xml::ElementEnumerator propert = _node->getElementEnumerator();
while (propert.next("Property"))
{
const std::string& key = propert->findAttribute("key");
const std::string& value = propert->findAttribute("value");
if (key == "TextureSize")
setTextureSize(utility::parseValue<IntSize>(value));
if (key == "TextureName")
setTextureName(value);
}
}

void RTTLayer::renderToTarget(IRenderTarget* _target, bool _update)
{
bool outOfDate = mOutOfDate || isOutOfDate();

if (outOfDate || _update)
{
MyGUI::IRenderTarget* target = mTexture->getRenderTarget();
if (target != nullptr)
{
target->begin();

for (VectorILayerNode::iterator iter = mChildItems.begin(); iter != mChildItems.end(); ++iter)
(*iter)->renderToTarget(target, _update);

target->end();
}
}

mOutOfDate = false;
}

void RTTLayer::setTextureSize(const IntSize& _size)
{
if (mTextureSize == _size)
return;

mTextureSize = _size;
if (mTexture)
{
MyGUI::RenderManager::getInstance().destroyTexture(mTexture);
mTexture = nullptr;
}

MYGUI_ASSERT(mTextureSize.width * mTextureSize.height, "RTTLayer texture size must have non-zero width and height");
std::string name = mTextureName.empty() ? MyGUI::utility::toString((size_t)this, getClassTypeName()) : mTextureName;
mTexture = MyGUI::RenderManager::getInstance().createTexture(name);
mTexture->createManual(mTextureSize.width, mTextureSize.height, MyGUI::TextureUsage::RenderTarget, MyGUI::PixelFormat::R8G8B8A8);

mOutOfDate = true;
}

void RTTLayer::setTextureName(const std::string& _name)
{
mTextureName = _name;

if (mTexture != nullptr)
{
IntSize size = mTextureSize;
mTextureSize.clear();
setTextureSize(size);
}

mOutOfDate = true;
}
ILayerItem* RTTLayer::getLayerItemByPoint(int _left, int _top) const
{
if (!mIsPick)
return nullptr;

#ifdef MYGUI_OGRE_PLATFORM
const MyGUI::IntSize& size = MyGUI::RenderManager::getInstance().getViewSize();
bool result = pickPositionInObject(_left, _top, size.width, size.height, mTextureSize.width, mTextureSize.height);
if (result)
{
return Base::getLayerItemByPoint(_left, _top);
}
#endif

return nullptr;
}

IntPoint RTTLayer::getPosition(int _left, int _top) const
{
if (!mIsPick)
return Base::getPosition(_left, _top);

#ifdef MYGUI_OGRE_PLATFORM
const MyGUI::IntSize& size = MyGUI::RenderManager::getInstance().getViewSize();
bool result = pickPositionInObject(_left, _top, size.width, size.height, mTextureSize.width, mTextureSize.height);
if (result)
{
mOldPoint.set(_left, _top);
}
#endif

return mOldPoint;
}
} // namespace MyGUI

SceneObject.h
/*!
@file
@author Albert Semenov
@date 12/2009
*/

#ifndef __SCENE_OBJECT_H__
#define __SCENE_OBJECT_H__

//#ifdef MYGUI_OGRE_PLATFORM

#include <Ogre.h>

#include "MyGUI_LastHeader.h"

//namespace demo
//{
class SceneObject
{
public:
SceneObject();
virtual ~SceneObject();

public:
void setEntity(const std::string& _value);
void setMaterial(const std::string& _value);
void setSceneManager(const std::string& _value);
void setCamera(const std::string& _value);

protected:
void setTextureName(const std::string& _name);

bool pickPositionInObject(int& _x, int& _y, int _view_width, int _view_height, int _texture_width, int _texture_height) const;

private:
// Code found in Wiki: http://www.ogre3d.org/wiki/index.php/RetrieveVertexData
void GetMeshInformation(
const Ogre::MeshPtr mesh,
size_t& vertex_count,
Ogre::Vector3* &vertices,
size_t& index_count,
unsigned long* &indices,
Ogre::Vector2* &coords,
const Ogre::Vector3& position,
const Ogre::Quaternion& orient,
const Ogre::Vector3& scale,
const std::string& _material);
void clear();

bool isIntersectMesh(int& _x, int& _y, const Ogre::Ray& _ray, int _texture_width, int _texture_height) const;
Ogre::Vector2 getCoordByTriangle(Ogre::Vector3 _position, const Ogre::Vector3& _corner0, const Ogre::Vector3& _corner1, const Ogre::Vector3& _corner2) const;
Ogre::Vector2 getCoordByRel(Ogre::Vector2 _position, const Ogre::Vector2& _corner0, const Ogre::Vector2& _corner1, const Ogre::Vector2& _corner2) const;

void updateData();

Ogre::SceneManager* getSceneManager() const;
Ogre::Camera* getCamera() const;

private:
Ogre::Vector2* mTextureCoords;
Ogre::Vector3* mVertices;
unsigned long* mIndices;
size_t mVertexCount;
size_t mIndexCount;
float mUScale;
float mVScale;
mutable Ogre::RaySceneQuery* mRaySceneQuery;
std::string mEntityName;
std::string mMaterialName;
std::string mTextureName;
std::string mSceneManager;
std::string mCamera;
Ogre::TextureUnitState* mTextureUnit;
};

//} // namespace demo

//#endif

#endif // __SCENE_OBJECT_H__

SceneObject.cpp/*!
@file
@author Albert Semenov
@date 12/2009
*/

#include <MyGUI.h>
#include "SceneObject.h"

//#ifdef MYGUI_OGRE_PLATFORM

//namespace demo
//{

SceneObject::SceneObject() :
mTextureCoords(nullptr),
mVertices(nullptr),
mIndices(nullptr),
mVertexCount(0),
mIndexCount(0),
mUScale(1),
mVScale(1),
mRaySceneQuery(nullptr),
mTextureUnit(nullptr)
{
}

SceneObject::~SceneObject()
{
clear();
if (mRaySceneQuery)
{
getSceneManager()->destroyQuery(mRaySceneQuery);
mRaySceneQuery = nullptr;
}
}

void SceneObject::GetMeshInformation(
const Ogre::MeshPtr mesh,
size_t& vertex_count,
Ogre::Vector3* &vertices,
size_t& index_count,
unsigned long* &indices,
Ogre::Vector2* &coords,
const Ogre::Vector3& position,
const Ogre::Quaternion& orient,
const Ogre::Vector3& scale,
const std::string& _material)
{
bool added_shared = false;
size_t current_offset = 0;
//size_t shared_offset = 0;
size_t next_offset = 0;
size_t index_offset = 0;

vertex_count = index_count = 0;

// Calculate how many vertices and indices we're going to need
for (unsigned short i = 0; i < mesh->getNumSubMeshes(); ++i)
{
Ogre::SubMesh* submesh = mesh->getSubMesh( i );
if (submesh->getMaterialName() != _material)
continue;

// We only need to add the shared vertices once
if (submesh->useSharedVertices)
{
if ( !added_shared )
{
vertex_count += mesh->sharedVertexData->vertexCount;
added_shared = true;
}
}
else
{
vertex_count += submesh->vertexData->vertexCount;
}

// Add the indices
index_count += submesh->indexData->indexCount;
}


// Allocate space for the vertices and indices
vertices = new Ogre::Vector3[vertex_count];
indices = new unsigned long[index_count];
coords = new Ogre::Vector2[vertex_count];

added_shared = false;

// Run through the submeshes again, adding the data into the arrays
for ( unsigned short i = 0; i < mesh->getNumSubMeshes(); ++i)
{
Ogre::SubMesh* submesh = mesh->getSubMesh(i);
if (submesh->getMaterialName() != _material)
continue;

Ogre::VertexData* vertex_data = submesh->useSharedVertices ? mesh->sharedVertexData : submesh->vertexData;

if ((!submesh->useSharedVertices) || (submesh->useSharedVertices && !added_shared))
{
if (submesh->useSharedVertices)
{
added_shared = true;
//shared_offset = current_offset;
}

const Ogre::VertexElement* coordElem =
vertex_data->vertexDeclaration->findElementBySemantic(Ogre::VES_TEXTURE_COORDINATES);

const Ogre::VertexElement* posElem =
vertex_data->vertexDeclaration->findElementBySemantic(Ogre::VES_POSITION);

Ogre::HardwareVertexBufferSharedPtr vbuf =
vertex_data->vertexBufferBinding->getBuffer(posElem->getSource());

unsigned char* vertex =
static_cast<unsigned char*>(vbuf->lock(Ogre::HardwareBuffer::HBL_READ_ONLY));

// There is _no_ baseVertexPointerToElement() which takes an Ogre::Real or a double
// as second argument. So make it float, to avoid trouble when Ogre::Real will
// be comiled/typedefed as double:
// Ogre::Real* pReal;
float* pReal;

for ( size_t j = 0; j < vertex_data->vertexCount; ++j, vertex += vbuf->getVertexSize())
{
posElem->baseVertexPointerToElement(vertex, &pReal);

Ogre::Vector3 pt(pReal[0], pReal[1], pReal[2]);

vertices[current_offset + j] = (orient * (pt * scale)) + position;

posElem->baseVertexPointerToElement(vertex + coordElem->getOffset() - posElem->getOffset(), &pReal);
Ogre::Vector2 coord(pReal[0], pReal[1]);
coords[current_offset + j] = coord;
}

vbuf->unlock();
next_offset += vertex_data->vertexCount;
}


Ogre::IndexData* index_data = submesh->indexData;
size_t numTris = index_data->indexCount / 3;
Ogre::HardwareIndexBufferSharedPtr ibuf = index_data->indexBuffer;

bool use32bitindexes = (ibuf->getType() == Ogre::HardwareIndexBuffer::IT_32BIT);

unsigned long* pLong = static_cast<unsigned long*>(ibuf->lock(Ogre::HardwareBuffer::HBL_READ_ONLY));
unsigned short* pShort = reinterpret_cast<unsigned short*>(pLong);


//size_t offset = (submesh->useSharedVertices)? shared_offset : current_offset;

// Ogre 1.6 patch (commenting the static_cast...) - index offsets start from 0 for each submesh
if ( use32bitindexes )
{
for ( size_t k = 0; k < numTris * 3; ++k)
{
indices[index_offset++] = pLong[k];/*+ static_cast<unsigned long>(offset)*/
}
}
else
{
for ( size_t k = 0; k < numTris * 3; ++k)
{
indices[index_offset++] = static_cast<unsigned long>(pShort[k]);/*+
static_cast<unsigned long>(offset)*/
}
}

ibuf->unlock();
current_offset = next_offset;
}
}

void SceneObject::clear()
{
if (mTextureCoords != nullptr)
{
delete mTextureCoords;
mTextureCoords = nullptr;
}

if (mVertices != nullptr)
{
delete mVertices;
mVertices = nullptr;
mVertexCount = 0;
}

if (mIndices != nullptr)
{
delete mIndices;
mIndices = nullptr;
mIndexCount = 0;
}
}

bool SceneObject::isIntersectMesh(int& _x, int& _y, const Ogre::Ray& _ray, int _texture_width, int _texture_height) const
{
Ogre::Real closest_distance = -1.0f;
Ogre::Vector3 closest_result;

// test for hitting individual triangles on the mesh
bool new_closest_found = false;
int index_found = 0;
for (int i = 0; i < static_cast<int>(mIndexCount); i += 3)
{
// check for a hit against this triangle
std::pair<bool, Ogre::Real> hit = Ogre::Math::intersects(_ray, mVertices[mIndices],
mVertices[mIndices], mVertices[mIndices], true, false);

// if it was a hit check if its the closest
if (hit.first)
{
if ((closest_distance < 0.0f) ||
(hit.second < closest_distance))
{
// this is the closest so far, save it off
closest_distance = hit.second;
index_found = i;
new_closest_found = true;
}
}
}

if (new_closest_found)
{
closest_result = _ray.getPoint(closest_distance);

// return the result
if (closest_distance >= 0.0f)
{
// raycast success
Ogre::Vector2 point = getCoordByTriangle(closest_result, mVertices[mIndices[index_found]], mVertices[mIndices[index_found+1]], mVertices[mIndices[index_found+2]]);
Ogre::Vector2 point2 = getCoordByRel(point, mTextureCoords[mIndices[index_found]], mTextureCoords[mIndices[index_found+1]], mTextureCoords[mIndices[index_found+2]]);

_x = (int)(point2.x * _texture_width);
_y = (int)(point2.y * _texture_height);

return true;
}
}

// raycast failed
return false;
}

Ogre::Vector2 SceneObject::getCoordByTriangle(Ogre::Vector3 _position, const Ogre::Vector3& _corner0, const Ogre::Vector3& _corner1, const Ogre::Vector3& _corner2) const
{
Ogre::Vector2 result; // ðåçóëüòàò

Ogre::Vector3 dirX = _corner1 - _corner0;
Ogre::Vector3 dirY = _corner2 - _corner0;

_position -= _corner0; // ðàññòîÿíèå îò íà÷àëà êîîðäèíàò (îò òî÷êè 0)

Ogre::Vector3 div = (dirX.crossProduct(dirY));
if (div.x != 0.0)
{
result = Ogre::Vector2((_position.crossProduct(dirY)).x, (dirX.crossProduct(_position)).x);
result /= div.x;
}
else if (div.y != 0.0)
{
result = Ogre::Vector2((_position.crossProduct(dirY)).y, (dirX.crossProduct(_position)).y);
result /= div.y;
}
else if (div.z != 0.0)
{
result = Ogre::Vector2((_position.crossProduct(dirY)).z, (dirX.crossProduct(_position)).z);
result /= div.z;
}
else
{
// ïèïåö
}

return result;
}

Ogre::Vector2 SceneObject::getCoordByRel(Ogre::Vector2 _position, const Ogre::Vector2& _corner0, const Ogre::Vector2& _corner1, const Ogre::Vector2& _corner2) const
{
Ogre::Vector2 result;

result = _corner0 + _position.x * (_corner1 - _corner0) + _position.y * (_corner2 - _corner0);

if (mUScale != 1)
{
float count = 1 / mUScale; // êîëëè÷åñòâî òàéëîâ
float x = result.x * count; // ïðîïîðöèåé óçíàåì ïîëîæåíèå
result.x = x + 0.5f; // ñìåùàåì íà ïîëîâèíó, ÷òîáû öåíòð òàéëà áûë â ñåðåäèíå
result.x = fmod(result.x, 1); // îòáðàñûâàåì äî çàïÿòîé ïîëó÷àÿ îò 0 äî 1
}

if (mVScale != 1)
{
float count = 1 / mVScale; // êîëëè÷åñòâî òàéëîâ
float y = result.y * count; // ïðîïîðöèåé óçíàåì ïîëîæåíèå
result.y = y + 0.5f; // ñìåùàåì íà ïîëîâèíó, ÷òîáû öåíòð òàéëà áûë â ñåðåäèíå
result.y = fmod(result.y, 1); // îòáðàñûâàåì äî çàïÿòîé ïîëó÷àÿ îò 0 äî 1
}

return result;
}

void SceneObject::setEntity(const std::string& _name)
{
mEntityName = _name;
updateData();
}

void SceneObject::setMaterial(const std::string& _material)
{
mMaterialName = _material;
updateData();
}

void SceneObject::updateData()
{
clear();

Ogre::Entity* entity = getSceneManager()->getEntity(mEntityName);
if (entity != nullptr && !mMaterialName.empty())
{
mVertexCount = 0;
mIndexCount = 0;
GetMeshInformation(entity->getMesh(), mVertexCount, mVertices, mIndexCount, mIndices, mTextureCoords, Ogre::Vector3::ZERO, Ogre::Quaternion::IDENTITY, Ogre::Vector3::UNIT_SCALE, mMaterialName);

Ogre::MaterialPtr material = (Ogre::MaterialPtr)Ogre::MaterialManager::getSingleton().getByName(mMaterialName);
if (!material.isNull())
{
mTextureUnit = material->getTechnique(0)->getPass(0)->getTextureUnitState("gui");
if (mTextureUnit)
{
mTextureUnit->setTextureName(mTextureName);
mUScale = mTextureUnit->getTextureUScale();
mVScale = mTextureUnit->getTextureVScale();
}
}
}
}

void SceneObject::setTextureName(const std::string& _name)
{
mTextureName = _name;

if (mTextureUnit != nullptr)
{
mTextureUnit->setTextureName(mTextureName);
}
}

bool SceneObject::pickPositionInObject(int& _x, int& _y, int _view_width, int _view_height, int _texture_width, int _texture_height) const
{
if (mRaySceneQuery == nullptr)
{
mRaySceneQuery = getSceneManager()->createRayQuery(Ogre::Ray());
}

Ogre::Ray ray = getCamera()->getCameraToViewportRay(
_x / float(_view_width),
_y / float(_view_height));

mRaySceneQuery->setRay(ray);
mRaySceneQuery->setSortByDistance(true);
Ogre::RaySceneQueryResult& result = mRaySceneQuery->execute();
for (Ogre::RaySceneQueryResult::iterator iter = result.begin(); iter != result.end(); ++iter)
{
if (iter->movable != 0)
{
if (iter->movable->getName() == mEntityName)
{
if (isIntersectMesh(_x, _y, ray, _texture_width, _texture_height))
{
return true;
}
break;
}
}
}

return false;
}

Ogre::SceneManager* SceneObject::getSceneManager() const
{
return Ogre::Root::getSingleton().getSceneManager(mSceneManager);
}

Ogre::Camera* SceneObject::getCamera() const
{
return getSceneManager()->getCamera(mCamera);
}

void SceneObject::setSceneManager(const std::string& _value)
{
mSceneManager = _value;
}

void SceneObject::setCamera(const std::string& _value)
{
mCamera = _value;
}

//} // namespace demo

//#endif

I just commented out some namespace calls and #if MyGUI_OGRE_PLATFORM setions (just the if clause, not the contents). I found two file versions of UnitTest_Layers, one from 2008 and one from 2009 - so I took the newer version. Is there a more recent version - or do I have to include anything else maybe?

Rendering the image on texture still works, the mouse on the rendered texture is ignored.

Kind regards,
Corak

To get the error tracked down I started to go through the code line by line.
Now I can figure out at least, if the mouse cursor is over the mesh or not (not very accurate, but at least is works). Right now I have no clue to get the mouse position inside the mesh/texture to trigger functions of widgets.

In SceneObject.cpp I found the function bool SceneObject::isIntersectMesh(int& _x, int& _y, const Ogre::Ray& _ray, int _texture_width, int _texture_height) const
{
Ogre::Real closest_distance = -1.0f;
Ogre::Vector3 closest_result;
// test for hitting individual triangles on the mesh
bool new_closest_found = false;
int index_found = 0;
for (int i = 0; i < static_cast<int>(mIndexCount); i += 3)
{
// check for a hit against this triangle
std::pair<bool, Ogre::Real> hit = Ogre::Math::intersects(_ray, mVertices[mIndices],
mVertices[mIndices], mVertices[mIndices], true, false);

// if it was a hit check if its the closest
if (hit.first)
{
if ((closest_distance < 0.0f) ||
(hit.second < closest_distance))
{
// this is the closest so far, save it off
closest_distance = hit.second;
index_found = i;
new_closest_found = true;
}
}
}

if (new_closest_found)
{
closest_result = _ray.getPoint(closest_distance);

// return the result
if (closest_distance >= 0.0f)
{
// raycast success
Ogre::Vector2 point = getCoordByTriangle(closest_result, mVertices[mIndices[index_found]], mVertices[mIndices[index_found+1]], mVertices[mIndices[index_found+2]]);
Ogre::Vector2 point2 = getCoordByRel(point, mTextureCoords[mIndices[index_found]], mTextureCoords[mIndices[index_found+1]], mTextureCoords[mIndices[index_found+2]]);

_x = (int)(point2.x * _texture_width);
_y = (int)(point2.y * _texture_height);

return true;
}
}

// raycast failed
return false;
}This loop >>>for (int i = 0; i < static_cast<int>(mIndexCount); i += 3)<<< insn't run a single time, as mIndexCount is 0. There seems to be no function of call to set mIndexCount to another value at all. However, if I set it manually to any other value, the program crashes.

Any ideas?

Put breakpoint into SceneObject::updateData() and check if data is actually taken from your mesh. If it is not then you can check which part wasn't found (i.e. mesh, material, texture pass or whatever is needed).

Hello Altren,
thank you for your further input We're getting close I hope.

I added a breakpoint in SceneObject::updateData() and retrieves all data (names of mesh, material for example). However, there is still no sign of the in-texture-coordinates. SceneObject::updateData() is calling GetMeshInformation, a void function:void SceneObject::GetMeshInformation(
const Ogre::MeshPtr mesh,
size_t& vertex_count,
Ogre::Vector3* &vertices,
size_t& index_count,
unsigned long* &indices,
Ogre::Vector2* &coords,
const Ogre::Vector3& position,
const Ogre::Quaternion& orient,
const Ogre::Vector3& scale,
const std::string& _material)
{
bool added_shared = false;
size_t current_offset = 0;
//size_t shared_offset = 0;
size_t next_offset = 0;
size_t index_offset = 0;

vertex_count = index_count = 0;

// Calculate how many vertices and indices we're going to need
for (unsigned short i = 0; i < mesh->getNumSubMeshes(); ++i)
{
Ogre::SubMesh* submesh = mesh->getSubMesh( i );
if (submesh->getMaterialName() != _material)
continue;
########### At this pont the for-loop is broken ##############
// We only need to add the shared vertices once
if (submesh->useSharedVertices)
{
if ( !added_shared )
{
vertex_count += mesh->sharedVertexData->vertexCount;
added_shared = true;
}
}
else
{
vertex_count += submesh->vertexData->vertexCount;
}

// Add the indices
index_count += submesh->indexData->indexCount;
}


// Allocate space for the vertices and indices
vertices = new Ogre::Vector3[vertex_count];
indices = new unsigned long[index_count];
coords = new Ogre::Vector2[vertex_count];

added_shared = false;

// Run through the submeshes again, adding the data into the arrays
for ( unsigned short i = 0; i < mesh->getNumSubMeshes(); ++i)
{
Ogre::SubMesh* submesh = mesh->getSubMesh(i);
if (submesh->getMaterialName() != _material)
continue;
Ogre::VertexData* vertex_data = submesh->useSharedVertices ? mesh->sharedVertexData : submesh->vertexData;


if ((!submesh->useSharedVertices) || (submesh->useSharedVertices && !added_shared))
{

if (submesh->useSharedVertices)
{
added_shared = true;
//shared_offset = current_offset;
}

const Ogre::VertexElement* coordElem =
vertex_data->vertexDeclaration->findElementBySemantic(Ogre::VES_TEXTURE_COORDINATES);

const Ogre::VertexElement* posElem =
vertex_data->vertexDeclaration->findElementBySemantic(Ogre::VES_POSITION);

Ogre::HardwareVertexBufferSharedPtr vbuf =
vertex_data->vertexBufferBinding->getBuffer(posElem->getSource());

unsigned char* vertex =
static_cast<unsigned char*>(vbuf->lock(Ogre::HardwareBuffer::HBL_READ_ONLY));

// There is _no_ baseVertexPointerToElement() which takes an Ogre::Real or a double
// as second argument. So make it float, to avoid trouble when Ogre::Real will
// be comiled/typedefed as double:
// Ogre::Real* pReal;
float* pReal;

for ( size_t j = 0; j < vertex_data->vertexCount; ++j, vertex += vbuf->getVertexSize())
{
posElem->baseVertexPointerToElement(vertex, &pReal);

Ogre::Vector3 pt(pReal[0], pReal[1], pReal[2]);

vertices[current_offset + j] = (orient * (pt * scale)) + position;

posElem->baseVertexPointerToElement(vertex + coordElem->getOffset() - posElem->getOffset(), &pReal);
Ogre::Vector2 coord(pReal[0], pReal[1]);
coords[current_offset + j] = coord;

}

vbuf->unlock();
next_offset += vertex_data->vertexCount;
}


Ogre::IndexData* index_data = submesh->indexData;
########### numTris = 0 ?!? ##############
size_t numTris = index_data->indexCount / 3;

Ogre::HardwareIndexBufferSharedPtr ibuf = index_data->indexBuffer;

bool use32bitindexes = (ibuf->getType() == Ogre::HardwareIndexBuffer::IT_32BIT);

unsigned long* pLong = static_cast<unsigned long*>(ibuf->lock(Ogre::HardwareBuffer::HBL_READ_ONLY));
unsigned short* pShort = reinterpret_cast<unsigned short*>(pLong);


//size_t offset = (submesh->useSharedVertices)? shared_offset : current_offset;

// Ogre 1.6 patch (commenting the static_cast...) - index offsets start from 0 for each submesh
if ( use32bitindexes )
{
for ( size_t k = 0; k < numTris * 3; ++k)
{
indices[index_offset++] = pLong[k];/*+ static_cast<unsigned long>(offset)*/
}
}
else
{
for ( size_t k = 0; k < numTris * 3; ++k)
{
indices[index_offset++] = static_cast<unsigned long>(pShort[k]);/*+
static_cast<unsigned long>(offset)*/
}
}

ibuf->unlock();
current_offset = next_offset;
}
}I added two annotations (####) at the lines the function skips data. How does GetMeshInformation works, do I pass the correct data?

Ok, here's my guess: it looks for submesh with "BrowserMaterial" material, but your mesh don't have such submesh, because you set material name through Ogre::Entity instance and it doesn't affect submeshes.
Try to remove this to lines to check my guess:
if (submesh->getMaterialName() != _material)
continue;

Altren, you are a Genius!!

Following your guess it is possible now to use the mouse input on textures, casted on that plane!

I don't dare to ask, but new question occurs : Is it possible to pass location and oriantation of the ParentSceneNode to the SceneObject, too? When the plane is not on (0/0/0) - or if its tilted/rotated, its still displayed correctly but the mouse injection is taken from (0/0/0) without tilt/rotate/translation. //entGround is the entity where the texture is applied on
entGround->setMaterialName("BrowserMaterial");
entGround->getParentSceneNode()->yaw( Ogre::Degree( 180 ) );
entGround->getParentSceneNode()->pitch( Ogre::Degree( -8 ) );
entGround->getParentSceneNode()->translate(-1,0,0,Ogre::Node::TS_PARENT);
Looking in SceneObject::updateData() I found the calling ofGetMeshInformation(entity->getMesh(), mVertexCount, mVertices, mIndexCount, mIndices, mTextureCoords, Ogre::Vector3::ZERO, Ogre::Quaternion::IDENTITY, Ogre::Vector3::UNIT_SCALE, mMaterialName);I tried to chance that call to GetMeshInformation(entity->getMesh(), mVertexCount, mVertices, mIndexCount, mIndices, mTextureCoords, entity->getParentSceneNode()->getPosition(), entity->getParentSceneNode()->getOrientation(), entity->getParentSceneNode()->_getDerivedScale(), mMaterialName);following Brocans approach in this thread on our board - but nothing changed.

Am I overseeing anything - or is there an other / working alternative, how to translate/rotate/roll objects with MyGUI-textures on it?

You should use _getDerivedPosition, _getDerivedOrientation and _getDerivedScale.

Hmm, I tried:GetMeshInformation(entity->getMesh(), mVertexCount, mVertices, mIndexCount, mIndices, mTextureCoords, entity->getParentSceneNode()->_getDerivedPosition(), entity->getParentSceneNode()->_getDerivedOrientation(), entity->getParentSceneNode()->_getDerivedScale(), mMaterialName);But the result is the same as before. I read out all x/y/z-Values and there are all 0.

Maybe I misunderstood you - and you meant _getDerivedPosition, _getDerivedOrientation and _getDerivedScale to be used in another place than in the GetMeshInformation function?

Here is how the object is attached (if it helps error tracking):mSceneMgr->getRootSceneNode()->createChildSceneNode()->attachObject(entGround);

Dear Altren,
finally I was able to solve the last step using your hint with the _getDerivedXyz.

The reason was: the function SceneObject::updateData() was called, before the transformation of the plane mesh was processed. I changed the order and now everything is working as intended.

Let's see how the system performs with multiple planes and more complicated meshs like spheres or curved planes (high-tech looging tech panels and HUDs).

Thank you very much for your time and all the help up to this point!

Kind regards,
Corak

Edit: Solved! (2nd RTT Layer item)