1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<TITLE> [Nel] NeL Status Update?</TITLE>
<LINK REL="Index" HREF="index.html" >
<LINK REL="made" HREF="mailto:corvazier%40nevrax.com">
<LINK REL="Previous" HREF="000409.html">
<LINK REL="Next" HREF="000406.html">
</HEAD>
<BODY BGCOLOR="#ffffff">
<H1>[Nel] NeL Status Update?</H1>
<B>Cyril Corvazier</B>
<A HREF="mailto:corvazier%40nevrax.com"
TITLE="[Nel] NeL Status Update?">corvazier@nevrax.com</A><BR>
<I>Fri, 20 Apr 2001 19:06:03 +0200</I>
<P><UL>
<LI> Previous message: <A HREF="000409.html">[Nel] NeL Status Update?</A></li>
<LI> Next message: <A HREF="000406.html">[Nel] CVS code/server renamed to code nelns</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#410">[ date ]</a>
<a href="thread.html#410">[ thread ]</a>
<a href="subject.html#410">[ subject ]</a>
<a href="author.html#410">[ author ]</a>
</LI>
</UL>
<HR>
<!--beginarticle-->
<PRE>Hi,
><i> E> Is there any way I can get an ETA on that 3DS plugin, and detailed
</I>documentation of how to structure the data for my world, so NeL will work
with it?
What you said about the NeL file format makes a lot of sense, and it match
what we planned to do in a near future.
Actually, i'm going to describe the current data management.
Today, our export plug-ins generate NeL binary files using the NeL
serialisation system.
The serialisation system is described in the document inserted at the end of
the mail. This document
will be available on the web site in the Doxygen Related Pages during the
next week.
Here is the URL of another document that describes how to build NeL 3d data
from your 3d editor and export them in NeL binary format:
<A HREF="http://www.nevrax.org/docs/doxygen/nel/3d_data_howto.html">http://www.nevrax.org/docs/doxygen/nel/3d_data_howto.html</A>
---
NeL Files and Serialisation
* Introduction
This is really quite a difficult subject to write about - so this file is an
introduction which describes the basic features and principles of our
system.
* How our files work in NeL
The NeL files are NOT designed to be man-readable. Interpretation and
generation of file contents is performed by the objects that are to be read
and written using a standardised mechanism. This mechanism was inspired by
the system provided by Java.
We use the term 'serialisable' to describe a class that can be read from/
written to a NeL data file.
Counter-intuitive as it may, at first, appear, each 'serialisable' class
supplies a single method that is used for both reading and writing.
Note that the files are encoded in little-endian and that the NeL library
code deals with conversion of endian-ness for big-endian platforms
* Serialisation beyond files
The serialisation system can be used for generating binary data buffers in
memory (without writing the result to a file) or for packing and unpacking
data for transfer over a LAN.
* How it works
Technically, we define a 'serialisable' class as a class that can be passed
to IStream::serial().
In order for a class to be serialisable it is sufficient for it to include
the following method:
void serial(IStream&).
The fact that we use a template method definition means that a serialisable
class does not have to be derived from any other class.
All standard types are serialisable due to a non-template prototypes shown
below.
STL containers of serialisable types are serialisadble
Pointers to non-polymorphic serialisable types are serialisable.
The IStream class definition looks something like this:
class IStream
{
...
void serial (int&);
void serial (float&);
...
template <class T> void serial (T&t)
{
t.serial (*this);
}
};
Example:
To make the following class serialisable:
class myFirstClass
{
int a,b;
};
you would need to extend the class as follows:
class myFirstClass
{
int a,b;
void serial (IStream&istream)
{
istream.serial(a);
istream.serial(b);
}
};
The following example shows how to serialise a more complicated data
structure
class myFirstClass
{
void serial (IStream&);
};
class myclass
{
int BaseType;
myFirstClass SerialisableClass
std::vector< myFirstClass> STLContainerOfSerialisableClass;
myFirstClass *PointerToSerialisableClass;
std::vector< myFirstClass*> STLContainerOfPointersToSerialisableClass;
void serial (IStream&istream)
{
istream.serial(BaseType);
istream.serial(SerialisableClass);
istream.serialCont(STLContainerOfSerialisableClass);
istream.serialPtr(PointerToSerialisableClass);
istream.serialContPtr(STLContainerOfPointersToSerialisableClass);
}
};
* Dealing with cross referenced or hierarchical data
If an object contains a pointer to another object in memory then the
serialPtr() method is used to read/ write the referenced object.
The NeL library code writes a value corresponding to the pointer to the
serialised data, followed by the data that the pointer points to (In the
case of a NULL pointer the value 0 is written without any following data)
The NeL library code automatically deals with the cases where two or more
objects reference the same object or there is a circular reference. Each
time a pointer is de-referenced, for writing, NeL checks against a table of
previous pointers; if the pointer value already exists in the table then no
data is written. At read time the data structures are faithfully
reconstructed.
* Dealing with polymorphism within cross referenced data
In a nut shell, in order to un-serialise a data record that one only has an
interface type for, one needs to store an additional identifier with the
data record that identifies it's real type. The mechanism for doing this is
best shown with an example:
class IBaseClass : public IStreamable
{
// This class is an interface. It is polymorphic.
virtual void foo ()=0;
// It must declare it's name
NLMISC_DECLARE_CLASS (MyClass);
};
class CClassToSerialise
{
IBaseClass *PointerToAPolymorphicClass;
void serial (IStream& s)
{
s.serialPolyPtr (PointerToAPolymorphicClass);
}
};
void main ()
{
...
// The polymorphic class must be registered in the registry
NLMISC_REGISTER_CLASS (MyClass);
...
}
* Dealing with file format evolution
void serial (IStream& s)
{
// At the begining of the serial process, read/ write the version number
of the class implementation
// In the following example - at read time 'version' contains the version
read from the stream. At
// write time version code '3' is written to the stream and to the
variable 'version'.
int version=s.serialVersion (3);
// Now switch the version
switch (version)
{
case 3:
// The last field added in the class
s.serial (LastField);
// do some different stuff at read time and write time
if (s.isReading())
{
// at read time
...
}
else
{
// at write time
...
}
case 2:
// note that the code provided as of here allows for the reading of old
versions of the class
s.serial (Toto);
// in the case where the evolution from my version 1 implementation to my
version 2
// is not simply an extension of version 1 we need to break execution
here
break;
case 1:
s.serial (Foo);
case 0:
s.serial (Truc);
}
}
* NeL File Headers
The objective of NeL file headers is to verify that a file is in the right
format before attempting to interpret the contents.
// The NeL team use the following advise serialise a file this way:
void CFileRootClass::serial (IStream& s)
{
// First write / read-check the header
s.serialCheck ((uint32)'_LEN');
s.serialCheck ((uint32)'HSEM');
// This code write / read-check the header 'NEL_MESH' at the beginning of
the file.
// If the check fails, serialCheck throws the EInvalidDataStream
exception.
}
* Good examples to look at:
include/nel/misc/stream.h // Stream base classes
class CTileBank in src/3d/tile_bank.cpp // Good example of file format
evolution
class CAnimation in src/3d/animation.cpp // Good example of polymorphism
---
Cyril Corvazier
Lead 3d programmer
Nevrax France
</pre>
<!--endarticle-->
<HR>
<P><UL>
<!--threads-->
<LI> Previous message: <A HREF="000409.html">[Nel] NeL Status Update?</A></li>
<LI> Next message: <A HREF="000406.html">[Nel] CVS code/server renamed to code nelns</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#410">[ date ]</a>
<a href="thread.html#410">[ thread ]</a>
<a href="subject.html#410">[ subject ]</a>
<a href="author.html#410">[ author ]</a>
</LI>
</UL>
</body></html>
|
