In this article I’ll show how to calculate per-vertex Normals and the Tangent Space. Here you’ll see the most accurate technique, generating real Normals and breaking the vertex when necessary. This article is an intermediate part of the “All about Shaders” series. Would be nice if you’ve read the first part All about Shaders – (part 1/3)

At a glance

First off, the calculations and routines that we’ll create here is not an easy task, there are complexes concepts and calculations involved here. So, be sure you have this macro view:

• Usually the 3D softwares will export an Optimized Per-Vertex Normal, for those cases we can save our time, avoiding re-create the Normals. So we’ll create the Normal Vector ONLY when the Normals from 3D file was not optimized or don’t exist.
• Non optimized Normals means that the same Normal vector was written many times by the 3D software. It happens in some 3D file formats, like COLLADA, that the same Normal vector can appear hundred times, making the parse processing very expensive.
• It’s a good idea to calculate the Tangent Space always as possible (all that we need is Vertex Position and Vertex Texcoord).
• The best way to deal with meshes using OpenGL is to use what we call “Array of Structures”, however I’ll show you a generice algorithym that can be used with “Structure of Arrays” as well.

If these four bullets sounds like “Greek*” for you, I highly recommend you read some other articles before proceed:

• All about OpenGL ES 2.x – (part 1/3)
• All about OpenGL ES 2.x – (part 2/3)
• All about OpenGL ES 2.x – (part 3/3)

(* well, if you are a greek guy, sorry for that and please, take this word as equivalent to “chinese” or something like that).

As I explained before, to calculate the Normals we just need the vertex position, but to calculate the Tangent Space we’ll need the texcoord already calculated. If the mesh we are working on doesn’t have texcoord we’ll skip the Tangent Space phase, because is not possible to create an arbitrary UV Map in the code, UV Maps are design dependents and change the way as the texture is made.

Hands at work!

Calculating Normals – Step 1

In theory, the technique to calculate the face normals is simple: We’ll find the perpendicular vector for each face (triangle). However as you saw in the first tutorial of “All About Shaders“, the face normals are not so good. So, we need to calculate the vertex normals.

The things become a little bit more complex when we try to calculate the vertex normals. Every Face Normal will affect the vertex that compose the face. One single vertex can be shared by multiple faces, so, the final Vertex Normal will be the averaged vector of each Face that share this Vertex. As each face has its own size, the averaged Vertex Normal vector should consider those differences.

Vertices shared by multiple faces will have the resulting Normal as an average of all adjacent faces' Normals.

Just with this concept, you can create the Normals, but they will seem strange in some meshes. Like this teapot at the left side. When I created my first Normals, I spent weeks trying to find what was wrong with my calculus or with this concept… “Everything is OK, but only that fucking vertex is not. Why?”, I thought. Many others said that was a problem with my code, a problem with my memory allocation, and all other kinds of shits. But only after look closely to a great 3D software I found the problem. I’ll show you the same image that I spent hours looking to until I find the solution.

Vertex Normals

Did you notice something strange? This is a basic teapot mesh, many 3D softwares give you this to make tests with materials and lights. But this mesh has ONE single strange vertex: a vertex with 2 Normal Vectors. Is it possible? Actually, NO. The mesh structure must follow a pattern, so you can’t have all vertices with 1 Normal and only one vertex with 2 Normals. This is a very important point that no one talk about it, actually, I never seen anyone talking about this.

It’s time to understand what happens. Your mesh structure is not complete until you calculate the Normals. Why? Because some vertices will be “break/split” into two or more vertices by the Normals. This is what happens in that image. The 3D softwares will not show you this, but there are two vertices, with the same position and texcoords, but with different Normals. Obviously the 3D softwares prefer to omit this for performance reasons, but we can’t omit this fact to OpenGL. We must inform to the Shaders that there are two vertices instead of a single one.

And how will we know where to break a vertex? By the angle between faces. WOW, this thing of Normals is becoming very complex! Yeah, I told you that this is a very important and complex part. Let’s think by steps. Usually the light looks continuous on a surface until an angle of ~80º (like a sphere), however two faces with an angle > ~80º will form a hard edge, like the table’s corners. OK, now translating to English, this is what we’ll do:

1. Calculate each face’s Normal.
2. Calculate the angle between face’s Normals.
3. At each group of ~80º we’ll create a new set of Normals.
4. Finally we’ll find the averaged Normal for each group, respecting the face’s size.

Well, now it looks more simple, with just 4 simple steps. Nice! OK, let’s put our hands on code. First off, let’s create our Vector/Math functions.

// Early in definitions...
typedef struct
{
	float x;
	float y;
} vec2;

// Vector's distance.
static inline vec2 vec2Subtract(vec2 vecA, vec2 vecB)
{
	return (vec2){vecA.x - vecB.x, vecA.y - vecB.y};
}

typedef struct
{
	float x;
	float y;
	float z;
} vec3;

static const vec3 kvec3Zero = {0.0f, 0.0f, 0.0f};

// Vector's length.
static inline float vec3Length(vec3 vec)
{
	// Square root.
	return sqrtf(vec.x * vec.x + vec.y * vec.y + vec.z * vec.z);
}

// Vector's normalization.
static inline vec3 vec3Normalize(vec3 vec)
{
	// Find the magnitude/length. This variable is called inverse magnitude (iMag)
	// because instead divide each element by this magnitude, let's do multiplication, it's faster.
	float iMag = vec3Length(vec);

	// Avoid divisions by 0.
	if (iMag != 0.0f)
	{
		iMag = 1.0f / iMag;

		vec.x *= iMag;
		vec.y *= iMag;
		vec.z *= iMag;
	}

	return vec;
}

// Vector's sum.
static inline vec3 vec3Add(vec3 vecA, vec3 vecB)
{
	return (vec3){vecA.x + vecB.x, vecA.y + vecB.y, vecA.z + vecB.z};
}

// Vector's distance.
static inline vec3 vec3Subtract(vec3 vecA, vec3 vecB)
{
	return (vec3){vecA.x - vecB.x, vecA.y - vecB.y, vecA.z - vecB.z};
}

// Checks for zero values.
vec3IsZero(vec3 vec)
{
	return (vec.x == 0.0f && vec.y == 0.0f && vec.z == 0.0f);
}

// The dot product returns the cosine of the angle formed by two vectors.
static float vec3Dot(vec3 vecA, vec3 vecB)
{
	return vecA.x * vecB.x + vecA.y * vecB.y + vecA.z * vecB.z;
}

// The cross product returns an orthogonal vector with the other two,
// that means, the new vector is mutually perpendicular to the other two.
static vec3 vec3Cross(vec3 vecA, vec3 vecB)
{
	vec3 vec;

	vec.x = vecA.y * vecB.z - vecA.z * vecB.y;
	vec.y = vecA.z * vecB.x - vecA.x * vecB.z;
	vec.z = vecA.x * vecB.y - vecA.y * vecB.x;

	return vec;
}

// Checks if there is a NaN value inside the informed vector.
// If a NaN value is found, it's changed to 0.0f (zero).
static vec3 vec3Cleared(vec3 vec)
{
	vec3 cleared;
	cleared.x = (vec.x != vec.x) ? 0.0f : vec.x;
	cleared.y = (vec.y != vec.y) ? 0.0f : vec.y;
	cleared.z = (vec.z != vec.z) ? 0.0f : vec.z;

	return cleared;
}

Now we’re ready to go further:

// Private variables...
// Assumes that all the variables that start with "_" is a private one
// and you must implement it by your self and some of those values must
// be present before you start:

// vec3 *_vertices    "array of vertices positions"   (required)
// vec3 *_texcoords   "array of texture coordinates"  (optional)
// vec3 *_normals     "array of normals"              (optional/to calculate)
// vec3 *_tangents    "array of tangents"             (to calculate)
// vec3 *_bitangents  "array of bitangents"           (to calculate)

// int  _vCount       "vertices count"                (required)
// int  _tCount       "texture coordinates count"     (optional)
// int  _nCount       "normals count"                 (optional/to calculate)
// int  _taCount      "tangents count"                (to calculate)
// int  _biCount      "bitangents count"              (to calculate)

// int *_faces        "array of face indices"         (required)
// int  _facesCount   "faces indices count"           (required)
// int  _facesStride  "stride of faces indices"       (required)

// Checks the crease angle for the normal calculations.
// This function creates and divides the normals for a vertex, recursively.
static unsigned int creaseAngle(unsigned int index, vec3 vector, vec3 **buffer, unsigned int *count, NSMutableDictionary *list)
{
	// Let's talk about this function later on.
}

// Calculating the Tangent Space.
void calculateTangentSpace()
{
	unsigned int i, length;
	unsigned int j, lengthJ;

	unsigned int *newFaces, *outFaces;
	unsigned int oldFaceStride = _facesStride;

	int i1, i2, i3;
	int vi1, vi2, vi3;
	int ti1, ti2, ti3;

	vec3 vA, vB, vC;
	vec2 tA, tB, tC;
	vec3 distBA, distCA;
	vec2 tdistBA, tdistCA;

	vec3 normal;
	vec3 tangent;
	vec3 bitangent;

	vec3 *normalBuffer;
	vec3 *tangentBuffer;
	vec3 *bitangentBuffer;

	NSMutableDictionary *multiples;

	Element *element;
	int vLength, vOffset;
	int nLength, nOffset;
	int tLength, tOffset;

	float area, delta;
	float *outValue;

	// Checks if the parsed mesh has Normals and Texture Coordinates.
	BOOL hasNormals = NO;// CUSTOM: Check if your mesh structure already has normals.
	BOOL hasTextures = YES;// CUSTOM: Check if your mesh structure already has texcoords.

	// Gets the vertex element.
	vLength = 4;// CUSTOM: Take the length of your vertex position component.
	vOffset = 0;// CUSTOM: Take the offset  of your vertex position component in the Array of Structures.

	// If the normal element doesn't exist yet, creates a new one.
	if (!hasNormals)
	{
		// CUSTOM: Create the Normal elements if it doesn't exist yet.
		// The element should be just as the vertex one, having a length and an offset.
		// IMPORTANT: Increate the stride of your Array of Structures (_facesStride).
	}

	// Gets the normal element.
	nLength = 3;// CUSTOM: Take the normal length.
	nOffset = 7;// CUSTOM: Take the normal offset. In this case (7) we consider it's after the texcoord.

	// If the texture coordinate element exist, gets it and create tangent and bitangent element.
	if (hasTextures)
	{
		tLength = 3;// CUSTOM: Take the texcoord length.
		tOffset = 4;// CUSTOM: Take the texcoord offset. In this case it's after the vertices positions.

		// CUSTOM: Here you should create the Tangent and Bitangent elements. Just as any
		// other element until here, they must have length and offset.
		// IMPORTANT: Increate the stride of your Array of Structures (_facesStride).
	}

	// Allocates memory to the new faces.
	newFaces = malloc(sizeof(int) * (_facesCount * _facesStride));

	// A priori, assumes that for each vertex exists only one normal.
	_nCount = _taCount = _biCount = _vCount;

	// Initializes the buffers for the tangent space elements.
	// This memory allocation must use calloc because they must be 0 (zero) value,
	// otherwise NaN values can be generated.
	normalBuffer = calloc(_nCount, sizeof(vec3));
	tangentBuffer = calloc(_taCount, sizeof(vec3));
	bitangentBuffer = calloc(_biCount, sizeof(vec3));

	// Initializes the dictionaries to deal with vertices with multiple normals in it.
	multiples = [[NSMutableDictionary alloc] init];

	// Loop through each triangle.
	length = _facesCount;
	for (i = 0; i < length; i += 3)
	{
		// Triangle Vertices. At this moment _faces is an ordered list of elements' indices:
		// iv1, it1, in1, iv2, in2, it2, iv3, it3, in3...
		//  |			   |			  |
		//  V			   V			  V
		// iv1,			  iv2,			 iv3
		// So the following lines will extract the indices of vertices that form a triangle.
		i1 = _faces[i * oldFaceStride + vOffset];
		i2 = _faces[(i + 1) * oldFaceStride + vOffset];
		i3 = _faces[(i + 2) * oldFaceStride + vOffset];

		// Calculates the vertex indices in the array of vertices.
		vi1 = i1 * vLength;
		vi2 = i2 * vLength;
		vi3 = i3 * vLength;

		// Retrieves 3 vertices from the array of vertices.
		vA = (vec3){_vertices[vi1], _vertices[vi1 + 1], _vertices[vi1 + 2]};
		vB = (vec3){_vertices[vi2], _vertices[vi2 + 1], _vertices[vi2 + 2]};
		vC = (vec3){_vertices[vi3], _vertices[vi3 + 1], _vertices[vi3 + 2]};

		// Calculates the vector of the edges, the distance between the vertices.
		distBA = vec3Subtract(vB, vA);
		distCA = vec3Subtract(vC, vA);

		//*************************
		//	Normals
		//*************************
		if (!hasNormals)
		{
			// Calculates the face normal to the current triangle.
			normal = vec3Cross(distBA, distCA);

			// Searches for crease angles considering the adjacent triangles.
			// This function also initialize new blocks of memory to the buffer, setting them to 0 (zero).
			i1 = creaseAngle(i1, normal, &normalBuffer, &_nCount, multiples);
			i2 = creaseAngle(i2, normal, &normalBuffer, &_nCount, multiples);
			i3 = creaseAngle(i3, normal, &normalBuffer, &_nCount, multiples);

			// Averages the new normal vector with the oldest buffered.
			normalBuffer[i1] = vec3Add(normal, normalBuffer[i1]);
			normalBuffer[i2] = vec3Add(normal, normalBuffer[i2]);
			normalBuffer[i3] = vec3Add(normal, normalBuffer[i3]);
		}
		else
		{
			// If the parsed file has normals in it, retrieves their indices in the array of normals.
			vi1 = _faces[i * oldFaceStride + nOffset] * nLength;
			vi2 = _faces[(i + 1) * oldFaceStride + nOffset] * nLength;
			vi3 = _faces[(i + 2) * oldFaceStride + nOffset] * nLength;

			// Retrieves the normals.
			vA = (vec3){_normals[vi1], _normals[vi1 + 1], _normals[vi1 + 2]};
			vB = (vec3){_normals[vi2], _normals[vi2 + 1], _normals[vi2 + 2]};
			vC = (vec3){_normals[vi3], _normals[vi3 + 1], _normals[vi3 + 2]};

			// Calculates the face normal to the current triangle.
			normal = vec3Add(vec3Add(vA, vB), vC);
		}

// CONTINUE...

Let's understand all those lines:

• Lines 1-16: Definition of the output variables.
• Lines 20-23: A very important function to calculate the crease angle between faces. We'll discuss more about it later on.
• Lines 28-59: Defining the variables we'll use to calculate the Normals and the Tangent Space.
• Lines 62-63: CUSTOM CODE. Checks if there are Normals and Texture Coordinates already calculated in your mesh structure.
• Lines 66-67: CUSTOM CODE. Gets the length of each Vertex set (usually 3 or 4) and its offset in the Array of Faces.
• Lines 70-75: CUSTOM CODE. If there is no previous calculated Normals, create a new element and define its length (usually 3) and its offset in the Array of Faces.
• Lines 78-79: CUSTOM CODE. Gets the length of each Normal set (usually 3) and its offset in the Array of Faces.
• Lines 82-90: CUSTOM CODE. If there is a calculated Texture Coordinates, then create the Tangent and Bitangent elements. As both Tangent and Bitangent will follow the same elements rule, so you just need to define the length and offset in face for one of them.
• Lines 93-106: Allocating the necessary memory for the calculations.
• Lines 109-110: Starting the loop through all triangles of your mesh.
• Lines 118-120: Getting the indices form the current working triangle.
• Lines 123-130: Getting the vertices related to the current triangle.
• Lines 133-134: Calculating the distance between triangle's vertices. Depending on the order which you calculate this distance, the resulting normal vector direction can change. So, be careful about changing this order.
• Lines 142-153: If there are no previous normals. The cross product between the calculated distances will generate a perpendicular vector, it's the face normal to the current triangle (Cross Product info). Then the crease angles are calculated, so we add the current normal vector to the normalBuffer. When we sum two vectors we are automatically calculating the middle vector. (Add Vector info).
  IMPORTANT: We're not normalizing the vector in this step, by doing so we make sure each triangle area (size) will be considered, because they have different lengths.
• Lines 158-168: If there are previous calculated normals. We calculate the current face normal, because the final format for normals are always vertex normal, not face normal. For the next steps we'll need the face normal.

The Crease Angle - Step 2

As I said before, the crease angle is a crucial part to create perfect normals, otherwise your normals will look correct for some meshes but very strange in other ones. My crease angle is recursive, that means, it will automatically break the vertex index and count as many times as necessary. So if one single vertex has 3 different normals, this function will break it 3 times.

This function must be called once per vertex. In the above code it's called at:
i1 = creaseAngle(i1, normal, &normalBuffer, &_nCount, multiples);
i2 = creaseAngle(i2, normal, &normalBuffer, &_nCount, multiples);
i3 = creaseAngle(i3, normal, &normalBuffer, &_nCount, multiples);

So the original vertex index can become a new index and this new one will be used in new normal vector and in tangent space (tangent and bitanget).

// Defines the maximum crease angle ~80.
#define kCreaseAngle		0.2

// Checks the crease angle for the normal calculations.
// This function creates and divides the normals to a vertex.
static unsigned int creaseAngle(unsigned int index,
								vec3 vector,
								vec3 **buffer,
								unsigned int *count,
								NSMutableDictionary *list)
{
	NSNumber *newIndex, *oldIndex;

	// Eliminates the NaN points.
	(*buffer)[index] = vec3Cleared((*buffer)[index]);

	// Checks if the informed normal vector is not zero.
	if (!vec3IsZero((*buffer)[index]))
	{
		// Calculates the cosine of the angle between the current normal vector and the
		// averaged normal in the buffer.
		float cos = vec3Dot(vec3Normalize(vector), vec3Normalize((*buffer)[index]));

		// If the cosine is greater than the crease angle, that means the current normal vector
		// forms an acceptable angle witht the averaged normal in the buffer. Otherwise, proceeds and
		// creates a new normal vector to the current triangle face.
		if (cos <= kCreaseAngle)
		{
			// Tries to retrieve an already buffered normal with the same bend.
			oldIndex = [NSNumber numberWithInt:index];
			newIndex = [list objectForKey:oldIndex];

			// If no buffer was found, create a new register to the current normal vector.
			if (newIndex == nil)
			{
				// Retrieves the new index and stores its value as a linked list to the old one.
				newIndex = [NSNumber numberWithInt:*count];
				[list setObject:newIndex forKey:oldIndex];
				index = [newIndex intValue];

				// Reallocates the buffer and set the new buffer value to zero, avoiding NaN.
				*buffer = realloc(*buffer, NGL_SIZE_VEC3 * ++*count);
				(*buffer)[index] = kvec3Zero;
			}
			// Otherwise, repeat the process with the buffered value to check for new crease angles.
			else
			{
				index = creaseAngle([newIndex intValue], vector, buffer, count, list);
			}
		}
	}

	return index;
}

Understanding line by line:

• Line 12: Clearing any possible NaN value. If a NaN value is found, it'll be set to 0.0f (zero).
• Line 15: Making sure that an empty buffer index will be skipped, because there is no reason to calculate the crease angle between a face and no other.
• Line 19: Calculating the "dot" product (Dot Product info) we get the cosine of the angle formed between two faces. These faces are: The new normal vector and the value that is already in the normal buffer. Both of them must be normalized at this point, otherwise their lengths will affect the resulting dot product.
• Line 24: Check if the calculated cosine is bellow the crease angle. So, if a cosine value is lower than the crease angle it means the calculated angle is higher than the maximum angle. Then the process of breaking the buffer index will start.
• Lines 27-31: Creating NSNumbers. The old value is, actually, the current index parameter. The line 28 tries to retrieve if there is an already calculated broken index referenced by the current index. If there is a previous calculated reference, that means, if the current index was broken before, then the crease angle function will be called again. Otherwise, the process of breaking the buffer index will continue.
• Lines 34-36: The new buffer index will be created in the end of the current buffer array. This new index will be stored and linked with the current index. So, if the current index appear again we already know where is it broken index.
• Lines 39-40: Reallocating the memory of the buffer array and setting the new index to zero, avoiding NaN values.
• Line 50: The returned index will always be one of these two situations:
  1. An empty buffer index (zero vector).
  2. A buffer index in which its stored normal forms an acceptable angle with the new normal.

Nice! Now we have the correct calculated Normals, respecting the crease angle. Besides if there is a previous calculated normals, this routine respects that and calculates only the face normal again.

Now it's time to enter in the Tanget Space.

Calculating the Tanget Space - Step 3

As you know the Tangent Space is formed by the Tangent Vector and the Bitangent Vector. Just to make this point clear, the word "Binormal" is wrong in the 3D context. Because a 2D circle can have a Binormal, however there is only one Normal Vector in the 3D space. So the perpendicular angle between the Normal and the Tangent is the Bitangent. Some guys still calling as Binormal, we can understand what they want to say, but we know this term is a misspelling, there is no Binormal in the 3D space.

OK, now let's understand the concept of the Tangent Space.

The Tangent Space is a local stuff related at how the light interacts on each face of the surface. But the tangent space doesn't exist in the reality, right? Right, it doesn't. The tangent space is something created as a workaround to use the "Bump Technique". Now a day, there are many bump techniques, like bump map, parallax map , displacement map and many others. Independent of the technique the tangent space vector must be calculated.

The tangent and bitangent are orthogonal vector with the Normal and instruct us about the direction of the face's texture coordinate (U and V map directions). This direction will be used to calculate the light based on the RGB colors of the bump image file.

So in short, the Tangent Space is just a convention that we create as a workaround the bump technique. Here is how we'll calculate the Tangent Space.

// CONTINUING...

		//*************************
		//  Tangent Space
		//*************************
		if (hasTextures)
		{
			// The crease angle process produces splits on the per-vertex normals, as the tangent space
			// must be orthogonalized, the tangent and bitanget follow those splits.
			if (_nCount > _taCount)
			{
				// Normals, Tangents and Bitangents buffers will always have the same number of elements.
				tangentBuffer = realloc(tangentBuffer, sizeof(vec3) * _nCount);
				bitangentBuffer = realloc(bitangentBuffer, sizeof(vec3) * _nCount);

				// Setting the brand new buffers to 0 (zero).
				lengthJ = _nCount;
				for (j = _taCount - 1; j < lengthJ; ++j)
				{
					tangentBuffer[j] = kvec3Zero;
					bitangentBuffer[j] = kvec3Zero;
				}

				_taCount = _biCount = _nCount;
			}

			// Retrieves texture coordinate indices.
			ti1 = _faces[i * oldFaceStride + tOffset] * tLength;
			ti2 = _faces[(i + 1) * oldFaceStride + tOffset] * tLength;
			ti3 = _faces[(i + 2) * oldFaceStride + tOffset] * tLength;

			// Retrieves the texture coordinates.
			tA = (vec2){_texcoords[ti1], _texcoords[ti1 + 1]};
			tB = (vec2){_texcoords[ti2], _texcoords[ti2 + 1]};
			tC = (vec2){_texcoords[ti3], _texcoords[ti3 + 1]};

			// Calculates the vector of the texture coordinates edges, the distance between them.
			tdistBA = vec2Subtract(tB, tA);
			tdistCA = vec2Subtract(tC, tA);

			// Calculates the triangle's area.
			area = tdistBA.x * tdistCA.y - tdistBA.y * tdistCA.x;

			//*************************
			//  Tangent
			//*************************
			if (area == 0.0f)
			{
				tangent = kvec3Zero;
			}
			else
			{
				delta = 1.0f / area;

				// Calculates the face tangent to the current triangle.
				tangent.x = delta * ((distBA.x * tdistCA.y) + (distCA.x * -tdistBA.y));
				tangent.y = delta * ((distBA.y * tdistCA.y) + (distCA.y * -tdistBA.y));
				tangent.z = delta * ((distBA.z * tdistCA.y) + (distCA.z * -tdistBA.y));
			}

			// Averages the new tagent vector with the oldest buffered.
			tangentBuffer[i1] = vec3Add(tangent, tangentBuffer[i1]);
			tangentBuffer[i2] = vec3Add(tangent, tangentBuffer[i2]);
			tangentBuffer[i3] = vec3Add(tangent, tangentBuffer[i3]);

			//*************************
			//  Bitangent
			//*************************
			// Calculates the face bitangent to the current triangle,
			// completing the orthogonalized tangent space.
			bitangent = vec3Cross(normal, tangent);

			// Averages the new bitangent vector with the oldest buffered.
			bitangentBuffer[i1] = vec3Add(bitangent, bitangentBuffer[i1]);
			bitangentBuffer[i2] = vec3Add(bitangent, bitangentBuffer[i2]);
			bitangentBuffer[i3] = vec3Add(bitangent, bitangentBuffer[i3]);
		}

// CONTINUE...

Understanding line by line:

• Line 6: Just create tangent space if there is texture coordinates on the mesh.
• Lines 10 - 25: Checking if the "Crease Angle" routine changed the normals. If so, adjust the tangents and bitangents buffers to have the same size as the normals buffer.
• Lines 27 - 42: Calculates the direction of the UV coordinates and the area of the current face (triangle).
• Lines 47 - 64: Calculates the tangent vector for the current triangle and add this value into the tangents buffer.
• Lines 71 - 76: As the Tangent Space is formed by three orthogonal vectors, we can calculate the last one (bitangent) very easy just by calculating the cross product between the Normal and Tangent.

OK, that's all, it's done!
Now we have all that we need: Normals, Tangents and Bitangents. They all are pretty perfect now and we just need to bring their values back into the arrays format. You can make this last step as you wish, I'll show you here a way that I'm used to.

Updating the Original Arrays - Step 4

OK, as I told you at the start, we'll use the "Array of Structures" in our final format. So we need to create/update that array and its indices based on the new elements, which may include Normals, Tangents and Bitangents.

I'll rewrite the "calculateTangentSpace()" function just to you take a look at it without breaks/continues.

// CONTINUING...		

		// Copies the oldest face indices and inserts the new tangent space indices.
		outFaces = newFaces + (i * _faceStride);
		lengthJ = _faceStride;
		for (j = 0; j < lengthJ; ++j)
		{
			*outFaces++ = (j < oldFaceStride) ? _faces[i * oldFaceStride + j] : i1;
		}

		outFaces = newFaces + ((i + 1) * _faceStride);
		for (j = 0; j < lengthJ; ++j)
		{
			*outFaces++ = (j < oldFaceStride) ? _faces[(i + 1) * oldFaceStride + j] : i2;
		}

		outFaces = newFaces + ((i + 2) * _faceStride);
		for (j = 0; j < lengthJ; ++j)
		{
			*outFaces++ = (j < oldFaceStride) ? _faces[(i + 2) * oldFaceStride + j] : i3;
		}
	}

	// Commits the changes for the original array of faces. At this time, it could looks like:
	// iv1, it1, in1, ita1, ibt1, iv2, it2, in2, ita2, ibt2,...
	_faces = realloc(_faces, sizeof(int) * (_faceNumber * _faceStride));
	memcpy(_faces, newFaces, sizeof(int) * (_faceNumber * _faceStride));

	// Reallocates the memory for the array of normals, if needed.
	if (!hasNormals)
	{
		_normals = realloc(_normals, sizeof(vec3) * _nCount);
	}

	// Reallocates the memory for the array of tangents and array of bitangents, if needed.
	if (hasTextures)
	{
		_tangents = realloc(_tangents, sizeof(vec3) * _taCount);
		_bitangents = realloc(_bitangents, sizeof(vec3) * _biCount);
	}

	// Loops through all new values of the tangent space, normalizing all the averaged vectors.
	length = _nCount;
	for (i = 0; i < length; i++)
	{
		// Puts the new normals, if needed.
		if (!hasNormals)
		{
			normal = vec3Normalize(normalBuffer[i]);
			outValue = _normals + (i * 3);
			*outValue++ = normal.x;
			*outValue++ = normal.y;
			*outValue = normal.z;
		}

		// Puts the new tangent and bitangent, if needed.
		// Isn't necessary here the Gram–Schmidt Orthogonalization process, because all the vectors
		// of the tangent space are already orthogonalized in reason of the crease angle approach.
		if (hasTextures)
		{
			tangent = vec3Normalize(tangentBuffer[i]);
			bitangent = vec3Normalize(bitangentBuffer[i]);

			outValue = _tangents + (i * 3);
			*outValue++ = tangent.x;
			*outValue++ = tangent.y;
			*outValue = tangent.z;

			outValue = _bitangents + (i * 3);
			*outValue++ = bitangent.x;
			*outValue++ = bitangent.y;
			*outValue = bitangent.z;
		}
	}

	// Frees the memories.
	free(newFaces);
	free(normalBuffer);
	free(tangentBuffer);
	free(bitangentBuffer);

	[multiples release];
}

Let's understand more about those last lines:

• Lines 4 - 21: Using a pointer to insert the new face indices in the "newFaces" array, pointers is much faster than accessing the array by indices. These lines also respect the old face indices to place the new ones in the last positions.
• Lines 26 - 27: Updating the original "array of faces". As you can see in the comments, at this point the new "array of faces" will be as "index of vertex 1, index of texcoord 1, index of normal 1, index of tangent 1, index of bitangent 1, index of vertex 2, ...". Remember that in this article I'll not include the job of converting those array into one single "array of structures", however with this "array of faces" you can do it very easy.
• Lines 30 - 40: Preparing the Normals, Tangents and Bitangents to receive the calculated values.
• Lines 43 - 74: Updating the Normals, Tangents and Bitangents values using pointers.
• Lines 77 - 82: Clearing the allocated memory.

Conclusion

WOW, a lot of code in this article, don't you think? I'll try to simplify the steps, as you may notice was only 4 steps until here. So, here is all that you need:

1. Calculate the Normals or use the Face Normals you already has.
2. When you need to calculate the Normals you must calculate the crease angle and split the Normals according to this calculation.
3. Calculate the Tangent Space, which includes Tangent and Bitangent.
4. Update the original arrays, including the Normals, Tangents and Bitangents.

OK, my friends, now you have all that you need to construct your preferable array for OpenGL ("array of structures" or "structure of arrays").

In my next article will continue the series about shaders. Let's talk about shader some techniques advanced skills in OpenGL ES Shaders.

If you have any doubts, just Tweet me:

Tweet

See you soon!

Due to many emails I have been received on the last months, I decide to create a single article to explain a little bit about the Singleton classes (also known as Singleton Pattern). I’ll focus here on Objective-C and Cocoa approach, however I’ll talk a little bit generally, so you can take the concept to other languages as well.

Let’s start!

Just Give me the Code!

Ok, ok. For who of you that just want the final code without protraction, here is it, you can choose a Xcode template or the pure code:

Xcode template (Xcode 4)

A cool template for Xcode 4 that I created to use over my working machines.

1. Download the zipped file bellow.
2. Unzip and copy the “db-in” folder to “/Library/Developer/Xcode/Templates/File Templates”
3. Open an Xcode project and try to add a new file. You will see a category called “db-in” on the left side, under the iOS platform.

Download now
Xcode Singleton Class template
40kb

Pure code

Here is the code for your header and source files:

@interface MySingleton : NSObject

+ (MySingleton *) instance;

@end

@interface MySingleton()

// Make any initialization of your class.
- (id) initSingleton;

@end

@implementation MySingleton

- (id) initSingleton
{
	if ((self = [super init]))
	{
		// Initialization code here.
	}

	return self;
}

+ (MySingleton *) instance
{
	// Persistent instance.
	static MySingleton *_default = nil;

	// Small optimization to avoid wasting time after the
	// singleton being initialized.
	if (_default != nil)
	{
		return _default;
	}

#if __IPHONE_OS_VERSION_MAX_ALLOWED >= __IPHONE_4_0
	// Allocates once with Grand Central Dispatch (GCD) routine.
	// It's thread safe.
	static dispatch_once_t safer;
	dispatch_once(&safer, ^(void)
				  {
					  _default = [[MySingleton alloc] initSingleton];
				  });
#else
	// Allocates once using the old approach, it's slower.
	// It's thread safe.
	@synchronized([MySingleton class])
	{
		// The synchronized instruction will make sure,
		// that only one thread will access this point at a time.
		if (_default == nil)
		{
			_default = [[MySingleton alloc] initSingleton];
		}
	}
#endif
	return _default;
}

- (id) retain
{
	return self;
}

- (oneway void) release
{
	// Does nothing here.
}

- (id) autorelease
{
	return self;
}

- (NSUInteger) retainCount
{
    return INT32_MAX;
}

@end

At a glance

First off, Singleton is a Design Patterns, I know you know some patterns and also use some of them every day in your works. Like MVC (Model View Controller) which all of us use to create our App in Xcode, or also the Delegate Pattern, which Cocoa framework uses largely. So the Singleton is one of those patterns. There are hundreds of patterns, if you want to learn more about Design Patterns I suggest you this book:
http://www.amazon.com/Design-Patterns-Elements-Reusable-Object-Oriented/dp/0201633612

OK, now about Singleton Pattern. It’s simple the concept of instantiate (allocate the necessary memory) for a class once and share this instance with all other classes/functions that need it, making at this form, a single point of access.

There are many discussion about the Singleton, many criticisms and many enthusiasm. In my humble opinion, the Singleton Pattern is very useful to control/manage some global data. Like the user account, for example, specially when some data need a special treatment, like user imagem data.

Anyway, my objective here is just to show you how to create a real Singleton class, using the minimum code as possible and make it thread-safe. Remember that in Objective-C multithreading is always something that we must be aware and think about.

Coding the concept

In theory the code is very simple, instead of allocate and initialize the class, we just call a class method that returns the singleton instance. We can use this approach through all our application:

// Inside a ClassA, for example.
MySingleton *foo = [MySingleton instance];
[foo setSomeValue:1];

// Inside a ClassB, for example.
MySingleton *foo = [MySingleton instance];
[foo someValue]; // gets the value that was set on ClassA.

It’s important to remember that a singleton class SHOULDN’T BE RELEASED! Using the Cocoa logic, when you call “instance” you are not allocating nor copying any memory, so you shouldn’t release it.

Well, but the memory must be allocated for someone, right? Yes, the memory will be allocated and controlled by the “instance” method. Let’s talk about the singleton lock.

Singleton Lock

We create a kind of lock to make sure our instance will be initialized only once. Like this:

+ (MySingleton *) instance
{
	// Persistent instance.
	static MySingleton *_default = nil;

	if (_default == nil)
	{
		_default = [[MySingleton alloc] init];
	}

	return _default;
}

This code will work for 99,9% of the time for ALL kind of languages. However, there is a rare kind of situation which this code will fail. We’re talking about the multithreading behavior where two threads try to access the singleton instance AND the singleton is not initialized yet.

In a very rare situation, two threads can try to initialize a singleton at the same time.

Notice that initializing a class is something very fast, could be done in one or two microseconds, however for a multithreading application, this could be time enough to cause a redundant instantiation. As the singleton, usually, is never deallocated, the double memory allocation will represent a leak.

To solve this problem, all that we need is to make sure our singleton lock be thread-safe, that means, only one thread can make the initialization process. At this point, with Objective-C we have many options, let’s see the best one.

Thread Safe

The best and faster approach is using the GCD (Grand Central Dispatch). However the GCD is not available for all iOS versions, just for iOS 4 or later. So we must think in a solution that fits good for all. The “@synchronized” instruction is something that works for all iOS versions, it creates a kind of funnel/queue in which only one thread can run at a time.

The problem with “@synchronized” is that it’s very slow and could make threads run at a synchronized way. So, to avoid unnecessarily wasting time with the @synchronized, let’s create a way to make the things run faster after the singleton being initialized:

+ (MySingleton *) instance
{
	// Persistent instance.
	static MySingleton *_default = nil;

	// Small optimization to avoid wasting time after the
	// singleton being initialized.
	if (_default != nil)
	{
		return _default;
	}

#if __IPHONE_OS_VERSION_MAX_ALLOWED >= __IPHONE_4_0
	// Allocates once with Grand Central Dispatch (GCD) routine.
	// It's thread safe.
	static dispatch_once_t safer;
	dispatch_once(&safer, ^(void)
				  {
					  _default = [[MySingleton alloc] init];
					  // private initialization goes here.
				  });
#else
	// Allocates once using the old approach, it's slower.
	// It's thread safe.
	@synchronized([MySingleton class])
	{
		// The synchronized instruction will make sure,
		// that only one thread will access this point at a time.
		if (_default == nil)
		{
			_default = [[MySingleton alloc] init];
			// private initialization goes here.
		}
	}
#endif
	return _default;
}
@end

Notice that the “if (_default != nil)” is a single check, that runs very very fast and can avoid our code of spending time with unnecessary checks. About the GCD and blocks, it’s a simple piece of code that makes the same thing as the code without GCD. The only difference is that the GCD approach uses the special data type “dispatch_once_t” (a kind of BOOL) to check if the code have been executed before.

If you want more information about GCD and the blocks, here is it:
http://developer.apple.com/library/ios/#documentation/Cocoa/Conceptual/Blocks/Articles/00_Introduction.html
http://developer.apple.com/library/ios/#documentation/Performance/Reference/GCD_libdispatch_Ref/Reference/reference.html
http://developer.apple.com/library/ios/#featuredarticles/Short_Practical_Guide_Blocks/_index.html

Final adjustments

OK, we’re almost done. Now, just to make sure that any other class (including classes from Cocoa framework) will not change the retain count of our singleton class, we can override some methods:

- (id) retain
{
	return self;
}

- (oneway void) release
{
	// Does nothing here.
}

- (id) autorelease
{
	return self;
}

- (NSUInteger) retainCount
{
    return INT32_MAX;
}

The dealloc method is not necessary, because the singleton will never be deallocated. But remember to release and retain the memory for internal variables, by the way this is why @property and @synthesize exist.

Conclusion

Very well, my friends, that is all about Singleton Pattern for Obj-C. I hope you liked. Enjoy your singletons and feel free to ask anything you want.

Cya!

I’m feeling that I must to pay more than a single tribute to Steve Jobs, I want to disseminate this tribute to more than a single post. So this is the deal, I want to talk a little bit about my idea with this tribute and start doing this.

The Image

This image, created by a student (Jonathan Mak Long) early this week became a symbol, an epic symbol and at the same time a tribute to Steve Jobs.

Tribute created by Jonathan Mak Long

This image is so amazing, it have hundreds of implicit significations and many subliminal values. But I don’t want to start a philosophical discuss about this. Just one thing I want to talk about is the first thing that this image make me remember.

I started to search how this image could be created and I found these two images:

They are good as well, but still far from the Long’s image. So, I found this another one:

Good, this image is more closest to what the Long’s image brings to my mind: Steve Jobs on the stage. It’s the strongest memory in my mind. Everyone knows how advanced was his rhetoric skills, how many implicit values and messages Jobs could put in one single sentence. His words reflect his way to think and his way changed everything. So, the great visionary that Steve Jobs was were reflected in their presentations and the Long’s image makes me remember that.

Finally, I found another image, a real photo, this one:

So, the Long’s image is the next step in this chain. I don’t know if the Jonathan Mak Long had this idea in his mind, of also represent the Jobs’ life on the stage. The point is that it comes to my mind when I saw the Long’s image. So, the new “Apple Logo” from Jonathan Mak is absolutely amazing under all points of view, I want to use it!

The Text

I heard this week: “We lost our john Lennon, we lost our hero”. This is true. Today all my life is around what the Steve Jobs did. I have macs on my work and at home, my phone is Apple, I play games on iPads, the last gift I gave to my wife was an iPod, we watch movies on an AppleTV, I teach Apple technology, everyday in my life I work/think in iOS APPs.

The changes that Steve Jobs made goes much beyond the technology. The Jobs’ legacy opened some whole new markets, with thousands companies working on it around the globe everyday, by the way, my daily job is on one of those companies. Steve Jobs really made millions of jobs around the world.

One of my first advertisements. (Click to enlarge)

5 years ago I was an ActionScript developer (Flash), but I hated that, because we can’t create great 3D on Flash (even today with Stage3D/Molehill, Flash developers can’t do the same as iOS can). With iOS I found myself! I found a place to put everything I know, to make great animations, great graphics, to think psychologically in the content and produce amazing user experiences. And the truth is, without Jobs, I could never find my place in this world.

We felt ourselves comfortable knowing we could focus on what we love, because there was one person on the backstage, that was thinking in “one more thing” that will change the day of tomorrow again. But now, this guy is dead…

But he showed to us the path, now we all know that everyone can change the world if they are “crazy enough to think they can change the world”. I want “to stay hungry, to stay foolish”. The Jobs death is not the end, it’s just the beginning of a new era, as he was used to say: a dot that we’ll connect only in some years when we look back, it’s impossible to understand the magnitude of this dot right now. The Jobs’ death is a dot in the history of the mankind.

The Tribute

So I want to do something, more than a simple tribute to Steve Jobs, I want to change something in my life now that will reflect my gratitude to the Jobs’ legacy. So, here is the symbol of this change, here is the deal:

From now and forever in my life, every post, every topic, every article, every single tutorial I make, I’ll finish with the following signature.

Let’s start a new series of tutorials, at this time let’s go deep in shaders universe, the most exciting part of OpenGL programmable pipeline. We’ll treat about textures, lights, shadows, per-pixel effects, bump, reflections and more.

This series is composed by 3 parts:

• Part 1 – Basic concepts about GLSL ES (Beginners)
• Part 2 – Shaders Effects (Intermediate)
• Part 3 – Mastering effects with OpenGL Shader Language (Advanced)

Here is a little list of contents to orient your reading:

At a glance

We’ll study in-depth the shaders language (more specifically the GLSL ES, the shader language for Embedded Systems) and let’s make great effects using the shaders like specular lights and reflections, bump maps, refractions, reflections and more.

In this first part I’ll cover the basic concepts sobre shaders and GLSL ES. We’ll need to drill deep in something called Tangent Space, which is an intermediate level, so I’ll create an article between the part 1 and 2 specific to treat Tangent Space concepts and its creation.

In the second part let’s start creating some interesting effects like specular lights, reflections and refractions. Besides, on the second one let’s create the environment mappings by using the cube texture.

Finally in the last part let’s make the most advanced effect, the bump mapping and see what is the difference between Normal Mapping, Bump Mapping and Parallax Mapping.

Hands to work!

Shading Types

top
First off, we need to understand the evolution of that we call shader. Today we have many computations on the GPU and several shader techniques that achieve really good results, but how did we get here?

Once upon a time, there was a single shading technique, called Flat Shading. It defines that light is computed with the normal vector and each FACE of the mesh has a normal vector. The term “FACE” here means a polygon (usually a triangle).

Flat Shading

With the evolution of the hardwares we start to process values for each vertex, this improvement took us into a new level, making more light effects. This shading technique was called Goraud Shading.

Goraud Shading

Then we discovered something that produces really nice results, the interpolation. It’s a process which we calculate all the intermediate points between two other points. We use it all the time. For example, if a point A has a texture coordinate U:1 V:2 and a point B has U:2 V:5, the interpolation will calculate all the middle points from 1,2 to 2,5. This is know as Phong Shading.

Phong Shading

But for many reasons, to simulate the real world, we need more than a linear interpolation. Surfaces have many details that change the way the light and shadow reacts. So we discovered a way to produce infinities values over a surface using less processing time. This technique was called Bump Shading.

Bump Shading

Today we have some advanced Bump’s techniques, but all of them have the same basic concept: store each value surface deformation value as a set of RGB color. So, basically, the Bump Shading takes advantage of a texture map that store coordinates within a RGB format. The coordinates in there can be used for many things: Normals, Tangents, Bitangents, Vertices positions or anything else we want. Usually the textures for Bump Shading is called Normal Maps, because we store the Normal Vector value in it.

By default, OpenGL uses the Phong Shading, making the interpolations between the Vertex Shader’s outputs and Fragment Shader’s inputs. This information is very important, so I’ll repeat: “Vertex Shader’s outputs are interpolated to Fragment Shader’s inputs”. Technically, this is what happens:

OpenGL Shaders

top
Here is a brief review about the shaders that we’ve seen on previous tutorials:

• Shader is the way to calculate everything related to our 3D objects by our own (from the vertices positions to the most complex light equations).
• Vertex Shaders (VSH) are processed one time for each object’s vertex. Fragment Shader (FSH) are processed one time for each fragment (not necessarely a pixel) of the visible object (http://db-in.com/blog/2011/01/all-about-opengl-es-2-x-part-13/).
• You can set constant values for Uniforms to work throughout the VSH and FSH processing (http://db-in.com/blog/2011/02/all-about-opengl-es-2-x-part-23/).
• Dynamic variables can be assigned only to the Attribute kind, which is exclusive of VSH. You can send a variable from VSH to FSH via Varyings, but remember that those values will be interpolated!

The shaders are small pieces of code that will be processed directly in the GPU. Unfortunately, even in these days (2011) our hardwares are very poor and slow if compared with the real amount of calculations that exist in the real world. The most advanced hardware trying to calculate a real phenomenon of the sun light passing through the water in a pool could take days to calculate a single frame. In the other hand, the real nature calculates all physical phenomenon instantly (OK, the mother nature makes a kind of “calculation”, not exactly an equation).

While we stay on the “Era of Bits” we can’t try to calculate the real phenomenons (I wrote an article about the “Binary World” where I talked about the new Era of Quantum computers, maybe there, in the “Era of Quantums”, we’ll be able to reproduce our 3D world closest from the reality).

Anyway, what I meant is that the shaders are something trying to reproduce the real world with a very very abstractly code, making a bunch of simplifications of the real world. So if you want to mastering the shader, you must learn to extract some abstract pieces of code from the real world phenomenons. But don’t worry too much about it now, in the right time you’ll see that it’s an easy task and can be cool as well.

It’s important to start thinking in the concept of “Shader Program”. It’s a set of 2 (and only 2) shaders: a vertex shader and a fragment shader. So, we must think in the render as 2 different steps (vertex and fragment). Usually the Fragment shader is processed a lot of times more than the Vertex one. If a mesh has 10.000 vertices that means its Vertex Shader will generate 10.000 outputs to the Fragment Shader. Remember that those outputs will always be interpolated to the Fragment processing. So, to increase the performance we always try to place hard calculus in the Vertex Shader. Obviously there are calculus that we can’t accept interpolation to their values, like the bump effect, just in these cases we make the calculus inside the Fragment Shader.

However, make sure you got the correct distinction between the concept of making the calculus inside the Vertex Shader and another thing called Per-Vertex/Per-Fragment Light. We’ll see those concept in-depth later on, but just to clarify:

• Per-Vertex Light means you have all the light calculus inside the Vertex Shader and them you interpolate the result to Fragment Shader.
• Per-Fragment Light means you have all last light calculus (the output value) inside the Fragment Shader, independent if the first steps was made in the Vertex or in the Fragment Shader.

The interpolation happens on all Vertex Attributes, like the Texture Coordinates. As shown above in the Interpolation Table, the interpolated values from a texture coordinate can retrieve all the pixels from a texture. The Texture Coordinates usually is defined by a technique called “UV Map” or “UV Unwrap Map”, which is an artistic job, actually is almost impossible to create detailed UV Maps only with the code. Often a professional 3D software export the Texture Coordinates values within the model coordinates, based on definitions from an user friendly UV editor.

But there is another per-vertex Attribute very important to 3D world. With the shaders we calculate lights, shadows, reflections, refractions and any other effects we want. All of them need something in common, a Normal vector.

Normal Vector

top
This is one of the most important per-vertex attributes and its concept is very easy to understand. In the real world, basically, there are two things that can alter how the light rays affect a surface: the material (reflectiveness, refraction, specularity, shininess, etc.) and the surface’s angle. Well, actually, the point of view (the viewer’s eyes) also affect how we see the light, but let’s focus on the first two things. The normal vector is related to the surface’s angle. As the performance is something crucial to us, instead to re-calculate the angle of each surface (triangle) at every shader processing, we are used to pre-calculate a normal vector to every surface (triangle).

In simple words, the normal vector is an unit vector (magnitude equals to 1.0 and all axis range vary between [0.0, 1.0]) which represents the surface’s angle. Nice, now, how we can calculate it?

Well, that’s not an easy task, my friend. I had to read/watch/try a bunch of tutorials until I found the right formula. There are many people trying to teach how to calculate the normals. Some say that you must calculate per-face normals and store them into a buffer, others say to calculate the averaged normals between adjacent faces, some even say that you need to calculate each surface area to include in your final calculation. But no one gave me the right formula! I had to find it by my self, with the help of a great 3D software called MODO (by the way, I love it!).

Unfortunately, I’ll not explain how to calculate the normals in this first tutorial. I’ll create a separated article to show you how to get the right formula to calculate the normals. The normals deserve more attention than a simple subject inside one tutorial.

The most valuable point here is you understand that the Normal is an unit vector and visualize how the normals work together and how they fit into our shaders’ context.

Normal’s Smooth Angle

top
As we always make an abstraction of the real world, trying to simplify it, we have created a concept that does not exist in the real world: the Smooth Angle. Imagine this: in the real world the surfaces has infinities vertices, take for example the image of an sphere, maybe a bowling ball.

Try to imagine the smallest face/triangle that composes that bowling ball. Even if we try to use a microscope, we never will see a faceted area. Now take a look at our virtual spheres, even if we create a 3D mesh using a stupidly high resolution of 8 millions polygons, we’ll stay very very far from the perfection of the real world. By the way, for our 3D applications and games, we must work around thousands polygons, not millions. Does that means our 3D lights will seem ugly on low meshes? Fortunately we have a solution.

Our 3D world always will have imperfections.

This problem can be solved with the Normal’s Smooth Angle. With simple words, it represents the maximum angle on which the light will looks continuous when reflected by a surface. The following picture helps us to understand this point better:

The Smooth Angle is the angle between two adjacent faces.

Remember that smooth angle should be calculated when we calculate the Normals, so any post-change to the smooth angle will affect the entire Normals. I’ll talk more about the smooth angle in the article of the Normals calculations.

Tangent Space

top
The Tangent Space is composed by two different components, actually there are three properties, the third one is the Normal Vector. As we’ve talked about the normals, let’s focus on the other two: Tangent and Bitangent (also known as Binormal, but the term “Binormal” is a mistake).

The Tangent and Bitangent are unit vectors, just as the Normal, the combination of these three components must form an Orthogonal and Orthonormal set. Before we go ahead, let me explain in simple words these two concepts:

• Orthogonal: Two vectors that are perpendicular (form an angle of 90 degrees).
• Orthonormal: A set of vectors that are all Orthogonals and unit vectors.

OK, these set of three vectors called Tangent Space is defined per-vertex. It’s purpose is define a local space for each face/vertex, which will be used to interpret the surface’s imperfection (bump map). The bump map (also known as normal map) is a RGB map that defines each relief of the surface.

It could sound confused to a text explanation. Just try to imagine this, as we always optimize everything in 3D world, the bump map is a technique that stores the surface’s deformations into a single image file. The Tangent Space is a set of vectors that allow us to parse the bump informations for each fragment, independent of the mesh’s rotation, position or scale. The following image illustrate the Tangent Space and its connection with the Texture Coordinates.

The Tangent Space is a set of 3 vectors.

Basically the Tangent Vector points to where the “S” coordinate increases (S Tangent) and the Bitangent points to where the “T” coordinate increases (T Tangent).

It’s possible to exist more than a Tangent Space for a single vertex, in this case the vertex will break into two or more vertices with the same value for position, actually there are other important concepts in Tanget Space, but I’ll not bother you with the details in here. Just as the Normal’s calculations, I’ll let this complex part to another article dedicated to that subject. The important thing here is to you understand what is the bump maps and how the Tangent Space is important to make bump effects.

Texture Coordinates

top
This is the most common vector component for shaders. It’s responsible to place an image on a mesh’s surface. The Texture Coordinates (texcoord for short), is defined per-vertex. It’ll be interpolated along two vertices to achieve a per-fragment result. The texcoord is usually given in the range [0.0, 1.0], representing the order [S, T], which are the normalized values from the [U, V] notation.

The texcoord have more to do with the artistic work than with our code, usually the 3D softwares are responsible for generating it. The texcoord will directly affects how the texture image file should be created, I mean, the image of the texture must be created based on the texcoord positions. There are some 3D softwares that accept multiple texcoord channels. It could be good for some situations which multiple designers are working together, but it’s not good to the performance and optimization. There is nothing that multiple texcoord channels can make that a single channel can’t. So, keep it simple, always try to work with a single texcoord channel.

The texcoord is very important to create the Tangent Space. Multiple texcoord channels will need multiple Tangent Spaces as well. So, multiple channels is never a good idea.

Conclusion

top
OK, my friends, I don’t want to make this first tutorial too long, so these are the basic concepts about the Shader. Now you know how the shaders work, what are their limitations, where is their power and what we need to have before enter in the shaders’ world.

In this tutorial you saw:

• The Shaders are responsible for all the visual results of our 3D world, including lights, shadows, reflections, refractions, etc.
• There are four shading techniques most used: Flat Shading, Goraud Shading, Phong Shading and Bump Shading. OpenGL by default will use the Phong Shading.
• The values from Vertex Shader to Fragment Shader will always be interpolated.
• We have 3 very important per-vertex vectors: Position, Normal and Texture Coordinate.
• The Normal Vector + 2 others form the Tangent Space, fundamental concept to produce the Bump Shading and any other displacement technique (like the Parallax Mapping).
• Always try to use only one texcoord channel.

Well done!
My next article will not be the second part of this series, instead, it’ll be a short article covering how to calculate the Normal Vector and the Tangent Space. We’ll need to have those vectors very correct before enter in the real calculus inside the shaders’ world.

If you have any doubts, just Tweet me:

Tweet

See you soon!

Due to some bugs and questions with the old tutorial, I’m creating this new one, much more simpler and less bugs than the another one. I’ll not post the old link here because everything you need to know you can find right here.

Nowadays, exist few alternatives to create a Framework to iOS, changing the default Xcode Script, which could not be a good choice if you want to publish the APPs constructed with your custom Framework. I’ll treat here about how to construct an Universal Framework to iOS, using the default tools from Xcode.

Let’s start!

Here is a little list of contents to orient your reading:

You can download the project I used in this article:

Download now
Xcode project files to Framework-iOS
403kb

FAQ

top
First off, I want to make sure you understand what this Framework to iOS can do, this can safe your time reading this article:

1. Can I use this Framework as a Bundle to store my files, XIBs, images?
   A: Yes, you can and now it’s very very easy to retrieve your files.
2. Can I use this Framework to import other Frameworks, like import UIKit, CoreGraphics, OpenGL?
   A: No. There is no way to do that. Your code in this custom Framework can import classes from other frameworks normally, but it is just a reference, classes from another framework will not be compiled at this time. So you must import the referenced Framework on the new project as well.
3. Will be my code visible to others?
   A: No. This Framework will export a compiled binary, so anyone can see inside it. You can make the same for some other files, like XIBs.
4. Why I need this?
   A: This is for developers/teams that want to share their codes without shows the entire code (.m/.c/.cpp files). Besides this is for who want to organize compiled code + resources (images, videos, sounds, XIBs, plist, etc) into one single place. And this is also for that teams that want to work together above the same base (framework).

Framework on iOS? Really?

top
Ok buddies, let’s make something clear, many people had said:”iOS doesn’t support custom Frameworks!”, “Custom Framework is not allowed at iOS!”, “Doesn’t exist custom Framework on iOS!” and many other discouraging things like these. Look, I’ve made many frameworks and worked with many others, I don’t believe that is really impossible to use a Framework on iOS. According to my experience and knowledge about Frameworks, it’s absolutely feasible a custom Framework on iOS Devices. If we think more about this issue we can find an elegant solution, right? First, let’s understand what a Framework really is, here is the definition of framework by Apple’s eyes:

A framework is a hierarchical directory that encapsulates shared resources, such as a dynamic shared library, nib files, image files, localized strings, header files, and reference documentation in a single package.

Doesn’t make many sense something with this description not be allowed in iOS, thinking in architecture and structure. Apple also says:

A framework is also a bundle and its contents can be accessed using Core Foundation Bundle Services or the Cocoa NSBundle class. However, unlike most bundles, a framework bundle does not appear in the Finder as an opaque file. A framework bundle is a standard directory that the user can navigate.

Good, now thinking about iOS security, performance and size, the only thing in a Framework definition which doesn’t fit in iOS technology is the “dynamic shared library”. The words “dynamic” and “shared” are not welcome in the iOS architecture. So the Apple allows us to work and distribute something called “Static Library“. I don’t like it! It’s not so easy as a Cocoa Framework, if a developer got your Static Library, he needs to set a header path too, or import the header files… it’s not useful, that’s a shit!

Well, so a Framework concept is absolutely compatible with iOS, except by the “dynamic shared library”, on the other hand Apple says that a “static library” is OK for iOS. So if we replace the “dynamic shared libraries” by a “static library” we could construct a Custom Framework to iOS, right?

Right!!!!

This is exactly what we’ll make in this article, let’s construct a Framework with Static Library, by the way, an Universal Framework.

Understanding Universal, Dynamic and Static concepts

top
Simple answer:

• Universal: Which works perfect on all architectures. iOS devices uses armv6 and armv7, iOS simulator on MacOS X uses i386.
• Dynamic: The compiler doesn’t include the target files directly. The classes/libraries are already pre-compiled (binary format) and lies on the system path. Besides, the dynamic libraries can be shared by many applications. This is exactly what Cocoa Framework is.
• Static: It represents that classes/libraries which is compiled by the compiler at the build phase. These files can’t be shared by other applications and relies on application path.

Simple as that. If you need more informations about Dynamic VS Static libraries, try this Apple’s Documentation.

No more concepts, hands at work!

Constructing a Framework Project

top

1. Create the Project:

top
I want to show you step by step of the entire process, so let’s start with the most basic, create an iOS project. You can choose one application template in Xcode, this is not really important, but remember to choose one template which could test your Framework code before export it.

Create an application project.

2. Framework Classes:

top

Create your framework classes.

Remember to create an “import header” to make everything simpler and organized to the user of your framework. Remember to write this header file with a framework notation, just as shown in the image bellow. Also remember to create your classes taking care to hide the classes which should not be visible to the other developers (users of your framework). We will set the public and private headers soon, but it’s very important to you protect the “core” classes, I mean, that classes which you don’t want to make visible to other developers.

For those private classes, you could import their header inside the “.m” (or .mm, .cpp, etc) file of a public class, by doing this you protect the header of private classes. Well, I know you probably already know that, I’m saying just to reinforce.

Remember that organization is 90% of a good framework, so try follow all the Apple advices to create your classes names, methods, properties, functions, etc.

Creating the Framework

top

3. Create a Framework Target:

top
OK, let’s create a target to compile our framework. Click on the icon of your project in the project navigator at the left and hit the button “Add Target”. A new window will come up. Now is our first trick. Instead to create a “Cocoa Touch Static Library” or a “Cocoa Framework” we will create a “Bundle” target.

A Bundle? Really? Yes! I can explain. A “Cocoa Framework” target can’t be compiled to armv6/armv7 and Xcode doesn’t allow us to use “Static Libraries” in a “Cocoa Framework”, so we can’t use this target. On the other hand, we can’t use “Cocoa Touch Static Library” either, because it doesn’t use the framework structure that we want.

Now, the Bundle target could be the best choice. It can hold any file we want, we can compile source code inside it and… we can turn it into a framework. To say the truth, almost all “Framework & Library” targets could be turned into a framework too, even the “Cocoa Touch Static Library”, throughout this article you probably will figure out how. For now, let’s create a Bundle target.

Create a Bundle target rather than Cocoa Touch Static Library.

4. Bundle Setup:

top
It’s time to make all the necessary changes to the Bundle target. Different than the old tutorial. You don’t need to clean up anything. Just know that everything else will be ignored (linked frameworks .plist files, .pch etc…).

I’m sure you already know this, but just to reinforce, here is the Build Setting screen, you can find it by clicking on the project icon in the left project navigator and then clicking in the “Build Setting” tab.

You must make a special Build Setting to turn a Bundle into a framework.

Here is our second great trick, or should be better to say “tricks”. Let’s change the “Build Setting” following this list:

• Base SDK: Latest iOS (iOS X.X) (in the X.X will appear the number of the lastest iOS SDK installed on your machine).
• Architectures: $(ARCHS_STANDARD_32_BIT) armv6 (it’s very important to be exactly this value including the space before “armv6″) This setting is valid to Xcode 4.2, if you are using an old version, use the “Standard (armv6 armv7)” option. (the values for this property depend on the value of the item bellow, so set that first).
• Build Active Architecture Only: NO (otherwise we can’t compile to armv6 and armv7 at the same time).
• Valid Architecture: $(ARCHS_STANDARD_32_BIT) (it’s very important to be exactly this value). If your Xcode is showing two lines with armv6 and armv7, delete then and insert this value in one single line.
• Dead Code Stripping: NO.
• Link With Standard Libraries: NO.
• Mach-O Type: Relocatable Object File. This is the most important change. Here, we instruct the compiler to treat the Bundle as a relocatable file, by doing this, we can turn it into a framework with the wrapper setting.
• Other Linker Flags: This setting is not mandatory, but if you are planning to use any kind of C++ code (.cpp or .mm) on this framework, Chris Moore (on the comments) advises to use the “-lstdc++” option. In this case could be a good idea to use “-ObjC” too, to avoid conflicts in old compilers.
• Wrapper Extension: framework. Here we change the Bundle to a Framework. To Xcode, frameworks is just a folder with the extension .framework, which has inside one or more compiled binary sources, resources and some folders, a folder, usually called Headers, contains all the public headers.
• Generate Debug Symbols: NO (this is a very important setting, otherwise the framework will not work on other computers/profiles).
• Precompile Prefix Header: NO.
• Prefix Header: “”. (Leave it blank).

IMPORTANT: Since the Xcode 4.x the architectures armv6 has no longer support. So, to create a real Universal Framework we must make a small “hack”:

1. After change the settings above close the Xcode, find the .xcodeproj (the project file) in Finder and then “Show Package Contents”.
2. Open the file “project.pbxproj” into a text editor.
3. Delete all the lines with VALID_ARCHS = “$(ARCHS_STANDARD_32_BIT)”.

5. Adding code and resources to the Bundle (Framework)

top
It’s time to place the content in our framework and define the public headers. To do that, with the Bundle target selected, click on the “Build Phase” tab. At bottom, hit the button “Add Phase” and then “Add Copy Headers“.

Open the recently created “Copy Headers” section and separate your public headers from the private or project headers. The difference here is:

• Public: Headers that other developers must know in order to work with your framework. In the final framework product, these headers will be visible even to Xcode.
• Private: Headers that is not necessary to other developers, but is good for consult or for reference. These headers will not be visible to Xcode, but will be in the framework folder.
• Project: Headers that the other developers nor Xcode have access. In reality these headers will not be placed in the final product, this is just to instruct the compiler to create your custom framework.

Now, open the “Compile Source” section and put there all your .m, .c, .mm, .cpp or any other compilable source file.

If your framework include not compilable files, like images, sounds and other resources, you can place them in the “Copy Bundle Resources” section. Later, when we generate the final framework, all your resources will be placed in a folder called “Resources”, but you can change it. That folder is very important, because it will be part of the path to retrieve your resources from the framework product.

Tip: To add many files at once, click on the “+” button and write the files’ extension on the search field. For example “.m”, “.c”, “.cpp”, “.h”, etc. This can save a lot of time.

This is how your “Build Phase” will looks like:

Define your compilable source and the headers.

Building the Universal Framework

top

6. Creating Universal Target:

top
To join both architectures products into one, we must to use the Lipo Tool. It’s a tool which comes with iOS SDK, just to know, it is in “/Platforms/iPhoneOS.platform/Developer/usr/bin”, its file name is “lipo”. But we don’t need to know this path, Xcode can deal with it to us.

Add a new target, hit the “Add Target” button, just as you did with Bundle Target. At this time a good choice is the “Aggregate” target. It doesn’t create any product directly, its purposes is just to aggregate another targets and/or run some scripts, exactly what we want! To use the Lipo Tool we’ll need to create a “Run Script” at the “Build Phase“.

Use the "Aggregate" target to construct a run script.

7. Lipo Tool Script:

top
This will be our greatest trick. The following script will make everything we need. It will compile the Framework Target to iOS Device and Simulator at once, will merge them with Lipo tool and will organize a good Framework Bundle structure.

Copy and paste this on your “Run Script” phase:

# Sets the target folders and the final framework product.
FMK_NAME=FI
FMK_VERSION=A

# Install dir will be the final output to the framework.
# The following line create it in the root folder of the current project.
INSTALL_DIR=${SRCROOT}/Products/${FMK_NAME}.framework

# Working dir will be deleted after the framework creation.
WRK_DIR=build
DEVICE_DIR=${WRK_DIR}/Release-iphoneos/${FMK_NAME}.framework
SIMULATOR_DIR=${WRK_DIR}/Release-iphonesimulator/${FMK_NAME}.framework

# Building both architectures.
xcodebuild -configuration "Release" -target "${FMK_NAME}" -sdk iphoneos
xcodebuild -configuration "Release" -target "${FMK_NAME}" -sdk iphonesimulator

# Cleaning the oldest.
if [ -d "${INSTALL_DIR}" ]
then
rm -rf "${INSTALL_DIR}"
fi

# Creates and renews the final product folder.
mkdir -p "${INSTALL_DIR}"
mkdir -p "${INSTALL_DIR}/Versions"
mkdir -p "${INSTALL_DIR}/Versions/${FMK_VERSION}"
mkdir -p "${INSTALL_DIR}/Versions/${FMK_VERSION}/Resources"
mkdir -p "${INSTALL_DIR}/Versions/${FMK_VERSION}/Headers"

# Creates the internal links.
# It MUST uses relative path, otherwise will not work when the folder is copied/moved.
ln -s "${FMK_VERSION}" "${INSTALL_DIR}/Versions/Current"
ln -s "Versions/Current/Headers" "${INSTALL_DIR}/Headers"
ln -s "Versions/Current/Resources" "${INSTALL_DIR}/Resources"
ln -s "Versions/Current/${FMK_NAME}" "${INSTALL_DIR}/${FMK_NAME}"

# Copies the headers and resources files to the final product folder.
cp -R "${DEVICE_DIR}/Headers/" "${INSTALL_DIR}/Versions/${FMK_VERSION}/Headers/"
cp -R "${DEVICE_DIR}/" "${INSTALL_DIR}/Versions/${FMK_VERSION}/Resources/"

# Removes the binary and header from the resources folder.
rm -r "${INSTALL_DIR}/Versions/${FMK_VERSION}/Resources/Headers" "${INSTALL_DIR}/Versions/${FMK_VERSION}/Resources/${FMK_NAME}"

# Uses the Lipo Tool to merge both binary files (i386 + armv6/armv7) into one Universal final product.
lipo -create "${DEVICE_DIR}/${FMK_NAME}" "${SIMULATOR_DIR}/${FMK_NAME}" -output "${INSTALL_DIR}/Versions/${FMK_VERSION}/${FMK_NAME}"

rm -r "${WRK_DIR}"

Now build the Aggregate target. Doesn’t matter you build for iOS Device or Simulator, this script will create a working folder, compile the framework target twice in there (device + simulator) and will output a folder called “Products” located in the project root folder. There is your Universal Framework to iOS!

Congratulations!

Importing your Universal Framework

top

8. Importing:

top
To test your Universal Framework, create a new Xcode project, select the Application target and go to “Build Phase” tab. Open the section “Link Binary With Libraries” and hit the “+” to add a new Framework. Click the “Add Other…” button and select your Universal Framework. Remember, you must to select the “.framework” folder. Remember to import your Framework Principal Header as a framework notation. Xcode will use your public headers in the Code Completion.

Let’s understand what happens until here:
When we set the “Mach-O Type” to “Relocatable Object File” the Xcode understand that everything related to that package will be like a “Binary Archive” like a ZIP. But that archive must be compiled again in new projects.

Then, when we create a Framework Bundle structure, Xcode understand that everything inside it is organized in folders like “Headers” and . But, as any other external bundle, to retrieve the resources you must load the external bundle. To iOS this could be an annoying step, however our Framework structure can help. Just Click and Drag on your Framework icon from “Project Navigator” to the “Copy Bundle Resources“. By doing this all the resources in your Framework will be copied to your Application Main Bundle.

Now, to retrieve the resources, make use of the main bundle, just as you are used to:

[[NSBundle mainBundle] pathForResource:@"FI.framework/Resources/FileName"
                                ofType:@"fileExtension"];

Conclusion

top
Well done, my friends! As we are used, let’s make a final review and take care with some possible problems.

• In a common Xcode project, create a Bundle target.
• Make the necessary setup, place your sources, headers and resources in it.
• Create an Aggreate target and place a Run Script in it.

One last advice: Take care with your classes structure. If you set, for example, the ClassB.h as a Project or Private header, but in your code you import it into a Public header, this will cause conflicts.

And one last tip: Notice in the sample project I removed the scheme for the “Bundle Target”. We don’t need that scheme any more, because the new script will manage the compilation to us.

That’s all, buddies.
Enjoy your Framework to iOS!

Thanks for reading,
See you soon!

After long months, just waiting for a single day, was a long wait, I know, but today the things will change a little bit. Today is THAT day.

After working through the nights, polishing every piece of code, thinking and rethinking routines… finally IT is here!

I’m very very happy to announce that NinevehGL is HERE!

Ladies and Gentlemen

With a great pleasure, let me introduce you the NinevehGL!

Keep it Simple

You know me, I’m so bored with many web sites, softwares, 3D engines and technologies that make our life a hell with their docs (or the absence of docs), poor tutorials, complex setups, complex API or even worst… they are paid! God Dammit!!!

• So, let’s try something different, something simple! Starting with the web site: nineveh.gl, just it, simple and easy.
• What about tutorials or docs? Very simple: http://nineveh.gl/docs/tutorials/, video tutorials!
• Long time to learn? Maybe… what do you think about just 30 min? 10 videos, 3 min each one. Sounds good?

Awesome features

There are many cool features in NinevehGL. But I think 3 are the “Killer Features”:

• OpenGL ES 2.0 (Programmable Pipeline): It uses the newest OpenGL ES version. More power, faster, lighter, better and shaders! With NinevehGL you can use all the power of the shaders and programmable pipeline.
• Import directly from 3D softwares! NinevehGL doesn’t need plugins or special 3D formats to import your files! Make use of Wavefront OBJ files or COLLADA files (all 3D softwares export one of them). NinevehGL is ready to import them.
• Made with pure Objective-C (Obj-C)! Yes, as an iOS developer, when I use OpenGL I expect to see Obj-C code, not C++ or C. NinevehGL is purely made with the native iOS language. Classes and routines follow all the Apple/Cocoa Touch guidelines.

Well, another great thing about NinevehGL is that it’s FREE! A 3D engine for iOS totally FREE!

This is a short post just to tell you about this great news. I hope you like it.

See you, guys!

In this article, I’ll talk about the binaries. I’ll cover the most important concepts and operations, let’s see some examples of where and how to use binaries and also how you can use it in your daily work. If you are looking for a tutorial about binaries or a “binary tutorial”, you are in the right place.

Let’s delve into the binary world.

Here is a little list of contents to orient your reading:

At a Glance

The binaries are the most basic and fastest approach in any programming language. Why? Our computers, and I mean, any kind of processing machine, is composed by a board with electric circuits on it, a lot of them. Through these circuits, electrical impulses passes every time, by one single circuit can pass thousands impulses per second.

Representation of electrical impulses on a computer board.

The receptors of the impulses are prepared to receive only one kind of information: “has impulse or not”, it’s a single “YES or NO”. This kind of information is exactly what a bit is. A bit can has the values 1 or 0 (“YES or NO”). Very simple, right?

This is why the binaries are the fastest approach ever. Because it’s the most singular representation of an electrical impulse, which is the basic communication of the components of our hardwares. Everything what comes next in this article is just a little of organization above this kind of information. We’ll see how organize the bits to form a byte, data types, files and everything else in our virtual world!

After we understand the binary world, we’ll apply some mathematics on it and make simple operations using the BITWISE operators. Finally we go up to high level programming language and see how the binaries can be used in our daily work to improve our applications.

Bit and Byte

top
Some people get confused with these names, so let’s make everything absolutely clear: “A Bit IS NOT a Byte!”. To our high level programming language, a single bit is “invisible”, we can’t set a bit individually, all our work is done on the byte level. To us, the byte is the most basic unit, not a bit. But what the hell is the diference between bit and byte?

Well, one byte is composed by 8 bits, this is a constant, it’s a rule, this never will change. 1 Byte = 8 Bits.

So, if we think in a sequence of 8 bits, remembering from combinatorial analysis, we can say that a set of 8 bits (with 2 possible values: 0 or 1) results in 256 possibilities (2^8 = 256). To make full use of these 256 possibilites, we can think in that sequence as Power of Two.

Every sequence of 8 bits forms 1 byte, which has 256 combinations.

Confused? Well, look at the next list, it could help you to understand this concept:

To compute the value, you look at every bit and check if it is valid or not (0 or 1), if it’s valid you sum the value it represents. Then you have the final value to a specific byte. The following image shows an example:

Each bit represents a single "YES" or "NO".

The calculation in the image above was: 128 + 64 + 32 + 4 + 1 = 229. Very simple, right?

Try it Yourself

top
You could try to discover the following values by yourself:

If you miss some value, don’t worry, because in reality we’ll never need to make this kind of calculation by our selves. As I told you before, what happens with bits inside a byte is invisible to our high level programming languages. We’ll never (or almost never) need to write “myVariable = 10101100;“, instead we’ll use: “value = 172;”

Now let’s go up one level and understand how a byte can be combined with another one to form more complex structures.

Combining Bytes

top
As we’ve saw a Byte can has 256 different values, 0 – 255. But we are used to use data types with a large range, like int, wich can be -2,147,483,648 to 2,147,483,647 or even the floating numbers, how it can be?

It’s simple, by placing more than 1 byte to describe a data. If you are 20 years or old, you probably remember from devices of 8 bits, right? Like the game console Nintendo 8 bits. Well, now you know, that console was capable to process data types only with 1 byte at time. After comes platforms of 16 bits, 32 bits and today we are used to work with 64 bits platforms. What that means?

In case of 16 bits, that means 1 byte can be processed together with another byte (2 x 8 bits), for a 32 bits platforms, that means 4 bytes can be processed together (4 x 8 bits), for a 64 bits platforms that means 8 bytes processed together (8 x 8 bits) and so on. The following picture illustrate 32 bits data types int and float:

Modern processors can work with multiple bytes at same time. This is a representation of 32 bits data types.

Thinking about it: a simple change of processing, from reading only 1 byte to more than one, can changes everything in our devices. For example, with 8 bits at once we have a range of 256 values for colors, do you remember from computers with that configuration? Yes, it was sucks! Now, with 4 bytes at the same time we can have systems running with 4,294,967,296 different colors. Imagine the performance and processing gain to our hardwares with that simple change: from reading 1 byte to reading many of them.

Just as a divagation, in these days, some people say about new kind of processors: Quantum Processors! The idea is change from electrical impulses on a board to electrons state in an atom. Basically an electron have 3 states in the energy level: Up (positive), Middle (neutral) or Down (negative), so instead only a single “YES or NO”, we could identify three states: “UP, MIDDLE and DOWN”. This single fact can drive us to a new perspective and a new kind of “bit”. Besides, unlike the impulses on a board, the electrons in an atom space don’t need to follow a physical path, its is just atomic vibration, imagine how fast it can be. Now imagine how many electrons can exist in a single atom, how many “bytes” (or something similar) we could process at the same time! Today some super computers are running on great platforms like 128 bits, 256 bits, 512 bits and more, but nothing could be compared to a Quantum Processor! Well, to say the truth, I’ve read that a team led by Yale University researchers has created the first quantum processor. This still far from the market, but not so far from our world.

Well, this is enough divagation. Back to the earth!

Great, now we know how our high level programming languages deal with binaries and more, we know how the binaries work. So every time you create an int data type, for example, in reality, you are creating a set of 32 bits, or in simple words, 32 slots of 0 or 1: 00000000 00000000 00000000 00000000.

It’s time to learn some operation we can make in the bit level.

Bitwise Operators

top
There are just few bitwise operators, actually 4. The actions they make are completely different than traditional mathematical operations.

   01001101
&  00101011
   --------
   00001001

   01001101
|  00101011
   --------
   01101111

   01001101
^  00101011
   --------
   01100110

~  01001101
   --------
   10110010

All the bitwise operators with use two sequences, just work for two sequences of the same length. But if one sequences has less bits, it will be filled with 0 at the left until reach the same length of the bigger sequence. For example: 01010100 & 111 is in reality the operation: 01010100 & 00000111. Remember that 0 bits on the left doesn't change anything, just as the real numbers.

Using these simple operations we can really make incredible routines. The best thing is that binaries are stupidly fast with any language, but we already talked about that. Soon we'll see some real examples of binaries in our day work. Before it, we need to see the Bit Shift operations, they are like a complementation to Bitwise operators.

Bit Shifts Operators

top
Exist two kind of shifts: left and right. Simple as that. Using this bit shift you'll push the bits in a byte set, that means, if your data type has 1 byte, bits will be shifted inside only 1 byte, if your data type has 4 bytes, bits will be shifted inside these 4 bytes.

   01001101 <<
   --------
   10011010

   01001101 >>
   --------
   00100110

The Bit Shift operator should be always in the right side. You also can specify multiple bits to shift by placing the number to shift at the right of the operator. For example:

.
   01001101 << 4
   --------
   11010000

   01001101 >> 3
   --------
   00001001
.

Oh right, my friends, these are all the binary operators. Now with these very simple operations we'll make really great things, including in our daily work.

Real Applications to the Binaries

top
I want to show you examples that we are used to use in our daily-work, some examples that can boost your application's performance without hard re-coding, just simple changes. So my deal with you is: only real examples. Let's go.

Even/Odd Routine

top
Let's start with a very simple example. Imagine a routine that you need to fill lines in a table, you want to fill those lines in pairs, one colored line and the one with blank. A simple way would be to count the rows and on each odd number row you fill in with color and on each even number row you leave it blank. Then we could create a routine to check the result of a division by 2, but the fastest way is using binaries. A binary routine could be something like this:

.
// Assume the numberOfRows variable was set.
for (i = 0; i < numberOfRows; ++i)
{
    if (i&1)
    {
        // The "i" is an odd number.
    }
    else
    {
        // The "i" is an even number.
    }
}
.

I love this routine. It makes me think about all the implicit rules that are within binary numbers. In the example above, what we are actually doing is:

00000001 & XXXXXXXX

As only the first bit has a value equals to 1, the only way to create an odd number is by using that bit. So, by using the AND operator, we check if the current "i" number makes use of that bit or not. In affirmative case, the number is odd, otherwise, it will be an even number. Very simple!

Working With Colors

top
Binary + Colors is one of the most popular topics. Almost all Image Softwares make use of binary routines to transform the images. For example, Photoshop makes all its image effects with binary changes in the pixel level.

The first thing about the binary + colors is to understand what is an hexadecimal number. An hexadecimal value is represented by 0xN (always need "0x" before the number), where N has 16 different values: 0-9 + A-F. So the real values to an hexadecimal number is:

0x0 = 0
0x1 = 1
0x2 = 2
0x3 = 3
0x4 = 4
0x5 = 5
0x6 = 6
0x7 = 7
0x8 = 8
0x9 = 9
0xA = 10
0xB = 11
0xC = 12
0xD = 13
0xE = 14
0xF = 15

Now we can place two hexadecimal numbers side by side, one will work like our "unit place" and the other as our "decimal place". So the first has 16 possible values and the second one has other 16 values, totalizing 256 possible values. Wait, wait, wait... 256? It's a Byte, right? So we can use every combination of two hexadecimal numbers (0xNN) to represent a byte!

Pair of hexadecimal numbers.

Just to make this step clear, here is some values using double hexadecimal numbers.

0x00 = 0      (0 x 16 + 0)
0x10 = 16     (1 x 16 + 0)
0x20 = 32     (2 x 16 + 0)
0x21 = 33     (2 x 16 + 1)
0x22 = 34     (2 x 16 + 2)
0xA0 = 160    (10 x 16 + 0)
0xA9 = 169    (10 x 16 + 9)
0xAA = 170    (10 x 16 + 10)
0xAB = 171    (10 x 16 + 11)
0xF0 = 240    (15 x 16 + 0)
0xFF = 255    (15 x 16 + 15)

Well, by using 32 bits platforms, we can use 4 bytes to describe a color, if we think in a RGB spectrum + Alpha we can reserve 1 byte to each color channel (R,G,B,A). At this way, each color channel has 256 possible values, right? We can represent a byte with a double hexadecimal number in the form 0xNN. So we could represent a full pixel with RGBA informations with an hexadecimal combination like this 0xNNNNNNNN, or using a "channel representation" 0xRRGGBBAA.

As a legacy from old times, the alpha processing doesn't has a pattern, so each software made its own implementation, some make 0xRRGGBBAA, others make 0xAARRGGBB and others even use 0xRRGGBB + 0xAA. Until today we don't have an universal format to alpha, so let's focus only in 0xRRGGBB to our study purposes.

Now, to isolate each color channel is very simple:

.
unsigned int color = 0x99AA44

unsigned char redChannel = color >> 16;
unsigned char greenChannel = ( color >> 8 ) & 0xFF;
unsigned char blueChannel = color & 0xFF;
.

It's very simple understand what happened. If we think in a binary representation (bits), the "color" variable above could be something like: RRRRRRRRGGGGGGGGBBBBBBBB. Then, when we move sixteen bits to the right, the resulting binary is:

RRRRRRRRGGGGGGGGBBBBBBBB >> 16 = RRRRRRRR

Similar we move eight bits to get the green channel, but at this time the green channel is not isolated yet, the red channel remains in the result. Then, to isolate the green channel we use the AND bitwise with 0xFF, which is a byte representing the real value 255, by doing so, we discard all the red channel information, remaining only with the green channel:

RRRRRRRRGGGGGGGGBBBBBBBB >> 8 = RRRRRRRRGGGGGGGG & 11111111 = GGGGGGGG

And finally we make almost the same to the blue channel:

RRRRRRRRGGGGGGGGBBBBBBBB & 11111111 = BBBBBBBB

If you get the pixel data from an image file, it could also has the alpha information pre-multiplied (ARGB). So to correctly extract the color channels you could make the & 0xFF step even to the red channel. If the alpha is post-multiplied (RGBA), we must to push 8 bits more:

.
unsigned int color = "one pixel data"

// To pixel data in format ARGB
unsigned char redChannel = ( color >> 16 ) & 0xFF;
unsigned char greenChannel = ( color >> 8 ) & 0xFF;
unsigned char blueChannel = ( color >> 0 ) & 0xFF;

// To pixel data informat RGBA
unsigned char redChannel = ( color >> 24 ) & 0xFF;
unsigned char greenChannel = ( color >> 16 ) & 0xFF;
unsigned char blueChannel = ( color >> 8 ) & 0xFF;
.

Once you have access to the color channel of each pixel you can make almost everything with the image color. You can exclude or replace a color channel, transform it into a grey scale image, change the colors individually, well... look at the Photoshop, all that effects was made with this principle. Obviously to produce more complex effects, like a blur or motion blur you need make many passes through every pixel and depending on the image size (width and height), this could be a very expensive task.

I've wrote about how to reduce the color range of an image using the binary approach. It was on my last OpenGL tutorial, you can check it by clicking here.

All the informations on this topic, binary + color, are about the most popular image formats, which make use of 8 bits per channel. But, there are some image format that support more, like 16 bits/channel or even 32 bits/channel. Those high formats support an incredible range of colors, they are often used to print formats, which uses CMYK instead RGBA color formats. But if you need to work with one of that "high images", the code is almost the same, just changing some numbers of bit shifts. To our virtual world, assume all images will always use 8 bits/channel. The following image shows the supported bits per channel for each of the popular image formats. Files that support 32 bits/channel also support 16 and 8, files that support 16 bits/channel also support 8:

Supported bits/channel on the most popular image formats.

Files are Bits

top
Everything was made with binaries. All the files that we use are in a low-level, only bits. So we can parse and extract information from any file. But to do that, you must know the bits organization in that file. This is the principle of file security and encryption. If you make a "coder" and a "decoder" to reorganize the bits inside a file, that file will be now encrypted. More sophisticated techniques also include fake bits or even create a "key" inside the file to recreate the file with a completely new structure. But in depth, everything are bits and can be read.

Usually the popular file formats have their bits organizations largely exposed on documents and specifications. So we can learn to read and write those formats.

Here is a list with the greatest website with file format specifications:

• http://www.fileformat.info
• http://www.wotsit.org
• http://www.martinreddy.net/ - for 2D image files
• http://www.martinreddy.net/ - for 3D format files

You also have 2 other choices: search in wikipedia for the file format you are looking for or even go to the page of the company/creator which made the file you are looking for. If you don't know who is the creator, you can go to this page: http://whatis.techtarget.com and search for the file extension, when you click you'll see a link to the official page of that format, if exist.

Talking about the file formats, in general, the great majority of the file creators try to organize the informations in 4 bytes, even those informations that need less than 4. This is a good practice to make the file more organized and more readable. But it's not a rule, some files could fill many informations into 4 byte.

Our high level programming languages often offer a class, or some classes, to work with binary files. So you can easily use any data type and that class will create a binary stream for you, forming a great array of bits. After put all informations on the files, you just save it, or send via internet, anyway. I'll show you a little piece of code using Objective-C (Cocoa):

.
// Creates some generic data to save into a file.
// As in C language the strings are made with an array of chars
// we must to know the length of the string to calculate
// the number of bits necessary to write it.
float floatNumber = 15.2;
char *charString = "this is a string";
int strLength = strlen(charString) + 1;

// Initializes the Cocoa class that deals with binaries.
NSMutableData *data = [[NSMutableData alloc] init];

// Inserts the data to save. This order is very important.
[data appendBytes:&floatNumber length:sizeof(float)];
[data appendBytes:&strLength length:sizeof(int)];
[data appendBytes:&charString length:sizeof(char) * stringLength];

// Saves the file.
[data writeToFile:@"/path/to/save/file.test" atomically:YES];

// Frees the memory allocated.
[data release];
.

Even if you are not familiar to Objective-C, you can understand the steps involved here. Using a standard C libraries, we also can use fopen function with the parameter "b" to open or write a file in the binary format.

The most important thing to understand is the order and length of each information in a binary file. In the example above, if we change the order of the calls to "appendBytes:length:" the resulting file will be absolutely different. The following image shows the resulting file from the code above and another file resulting from one little change in the code:

Every file is a stream of bits.

As you can see, changing the order of one little information drives us into a whole new file. The image above shows the binary file opened in an "Hexadecimal Editor". This kind of application show to us the binary format of any file, separating the data in 4 bytes. You remember how to represent a byte using hexadecimal numbers, right? 0xNNNNNNNN represents 4 bytes.

To read the binary file, you need follow the inverse path: selecting a set of bits by time and parsing it appropriately. For example, if you know the next 4 bytes represents a float data, parse it in a float data type, if you know the next byte represents a single char, parse it into a char and so on. Here is an important thing, to parse an information with mutable size, like a string, you need to know the length of the data in first place. So every time we need save a data with mutable size, we reserve some bits (usually 32 bits) to hold the length of the data that comes next.

Using the same example above, we could write to open the saved file:

.
// Creates the variables to receive the data from binary file.
float floatNumber;
char *charString;
int strLength;

// Creates a location/pointer to guide the position of the bytes inside the binary file.
int lastLocation = 0;

// Initializes the Cocoa class that deals with binaries.
NSData *data = [[NSData alloc] initWithContentsOfFile:@"/path/to/save/file.test"];

// Retrieves the floating number from the file and updates the location/pointer.
[data getBytes:&floatNumber range:NSMakeRange(lastLocation, sizeof(float))];
lastLocation += sizeof(float);

// Retrieves the string length from the file and updates the location/pointer.
[data getBytes:&strLength range:NSMakeRange(lastLocation, sizeof(int))];
lastLocation += sizeof(int);

// Allocates the necessary memory to receive the string and retrieves the data.
charString = malloc(sizeof(char) * strLength);
[data getBytes:&charString range:NSMakeRange(lastLocation, sizeof(char) * strLength)];

// Frees the memory allocated.
[data release];
.

Working with binary files we can make great thing on our application. We can improve the security, we can create binary files to hold informations or even open the binary files of other softwares. The performance and size is really amazing! Few kilobytes can hold a bunch of informations.

Conclusion

top
Well, as you can imagine now, we can use binaries to many things: to improve our application's performance, for security improvements, for read and write files, the binaries are useful for many things. My last advice is: don't be shy, use and abuse of binaries! After I learned about the binaries I waited many months until I really started to explore them. Don't do the same, explore the binary power right now, as in The Matrix (movie), you'll start to see the bits behind the world!

So, let's remember from everything:

• Bits are the most basic representation of the electrical impulses on our computer boards.
• A Bit can be 0 or 1.
• One Byte has 8 Bits. Totalizing 256 different possibilities.
• Bytes can be combined to form powerful platforms: 16 bits, 32 bits, 64 bits and so on.
• Bitwise operators (&, |, ^ and ~) generate new bits with some changes on the original bit(s).
• Bit Shifts operators (>> and <<) generate new bits changing the position of the original bit(s).
• One pair of hexadecimal values can represent one byte.
• Every file on any computer is a binary file and can be read if you know the bit sequence.

Very well my friends, this is all about the binaries. As you saw, they are really simple, very simple and in reason of this simplicity the binaries are so powerful. If you have any doubt, just ask.

Thanks for reading,

See you soon.

Is time to dive in advanced knowledges about OpenGL and 3D world. In this tutorial we’ll see many things about the 2D graphics, multisampling, textures, render to off-screen surfaces and let’s try to optimize at maximum our applications’ performance.

Is very important to you already know about all the concepts covered in the other 2 parts of this serie, if you miss something, here is a list:

This serie is composed by 3 parts:

• Part 1 – Basic concepts of 3D world and OpenGL (Beginners)
• Part 2 – OpenGL ES 2.0 in-depth (Intermediate)
• Part 3 – Jedi skills in OpenGL ES 2.0 and 2D graphics (Advanced)

At this time I imagine you know a lot of things about OpenGL and 3D world. You probably already created some applications using OpenGL, discovered many cool things, found some problems, even maybe your own engine/framework is under construction and I’m very glad in see you coming back.

As I’ve read once in a book: “One day you didn’t know to walk. Then you learned how to stand up and walk. Now is time to run, jump and swim!”… and why not “to fly”. With OpenGL our imagination has no limits, we can fly.

Let’s start.

Here is a little list of contents to orient your reading:

At a glance

Remembering everything until here:

1. OpenGL’s logic is composed by just 3 simple concepts: Primitives, Buffers and Rasterize.
2. OpenGL ES 2.x works with programmable pipeline, which is synonymous of Shaders.
3. OpenGL isn’t aware of the output device, platform or output surface. To make the bridge between OpenGL’s core and our devices, we must use EGL (or EAGL in iOS).
4. The textures are crucial and should have a specific pixel format and order to fit within OpenGL.
5. We start the render process by calling glDraw*. The first steps will pass through the Vertex Shader, several checks will conclude if the processed vertex could enter in the Fragment Shader.
6. The original structure of our meshes should never change. We just create transformations matrices to produce the desired results.

First I’ll talk about 2D graphics, then let’s see what is the multisampling/antialias filter, personally I don’t like the cost x benefit of this kind of technique. Many times an application could run nicely without multisampling, but a simple multisampling filter can completely destroy the performance of that application. Anyway, sometimes, it’s really necessary to make temporary multisampling to produce smooth images.

Later I’ll talk about textures in deep and its optimization 2 bytes per pixel data format. Also let’s see PVRTC and how to place it in an OpenGL’s texture, besides render to an off-screen surface.

And finally I’ll talk a briefly about some performances gains that I discovered by my self. Some tips and tricks which really help me a lot today and I want to share this with you.

Let’s go!

2D graphics with OpenGL

top
Using 2D graphics with OpenGL is not necessary limited to the use of lines or points primitives. The three primitives (triangles, lines and points) are good to use with 3D and 2D. The first thing about 2D graphics is the Z depth. All our work becomes two dimensions, excluding the Z axis to translations and scales and also excluding X and Y to rotations. It implies that we don’t need to use the Depth Render Buffer anymore, because everything we draw will be made on the same Z position (usually the 0.0).

A question comes up: “So how the OpenGL will know which object should be drawn on the front (or top) of the other ones?” It’s very simple, by drawing the objects in the order that we want (objects at the background should be drawn first). OpenGL also offers a feature called Polygon Offset, but it’s more like an adjustment than a real ordering.

OK, now we can think in 2D graphics at three ways:

1. Many squares on the same Z position.
2. Many points on the same Z position.
3. All above.

This is how a 2D graphics looks like for OpenGL.

How a 2D scene will appear on the screen.

You could imagine how easy it’s to OpenGL, a state machine prepared to work with millions of triangles, to deal with those few triangles. In extreme situations, 2D graphics works with hundreds of triangles.

With simple words, everything will be textures. So most of our work with 2D will be on the textures. Many people feel compelled to create an API to work with texture Non-POT, that means, work with texture of dimensions like 36 x 18, 51 x 39, etc. My advice here is: “Don’t do that!”. It’s not a good idea to work with 2D graphics using Non-POT textures. As you’ve seen in the last image above, it’s always a good idea work with a imaginary grid, which should be POT, a good choice could be 16×16 or 32×32.

If you are planing to use PVRTC compressed image files, could be good to use a grid of 8×8, because PVRTC minimum size is 8. I don’t advice make grids less than 8×8, because it’s unnecessary precise and could increase your work developing, also it compromise your application’s performance. Grids with 8×8 size are very precise, we’ll see soon the diferences between the grids and when and how to use them. Let’s talk a little bit more about the grid.

The Grid Concept

top
I think this is the most important part in the planing of a 2D application. In a 3D game, for example, to determine where a character can walk we must create a collision detector. This detector could be a box (bounding box) or a mesh (bounding mesh, a simple copy from the original). In both cases, the calculations are very important and expensive. But in a 2D application it’s very very easy to find the collisions areas if you are using a grid, because you have only a square area using X and Y coordinates!

This is just one reason because the grid is so important. I know you can come up with many other advantages of the grid, like the organization, the precision on the calculations, the precision of the objects on the screen, etc. About 10 year ago (or maybe more) I worked with a tiny program to produce RPG games with 2D graphics. The idea of grid was very well stablished there. The following images show how everything can be fitted into the grid:

Grid in the RPG Maker. (click to enlarge)

I know, I know… it’s the “Fuckwindons” system, sorry for that… as I told you, it was a decade ago… OK, let’s understand the important features about the grid. You can click on the side image to enlarge it. The first thing I want you notice is the about the overlap of the images. Notice that the left side of the window is reserved to a kind of library. At the top of this library you can see little square images (32×32 in this case). Those squares are reserved to the floor of the scenario (in our OpenGL language, it would be the background). The other images in that library are transparent images (PNG) wich can be placed on top of the floor squares. You can see this difference looking at the big trees placed on the grid.

Now find the “hero” on the grid. He’s at the right side near a tree, it shows a little face with redhead inside a box. This is the second important point about the grid. That hero doesn’t occupies only one little square on the grid, he could be bigger, but to the grid, the action delimitator represents only one square. Confused? Well, it’s always a good idea to use only one grid’s square to deal with the actions, because it let your code much more organized than other approaches. To create areas of actions you can duplicate the action square, just like the exit of the the village on the right top area of the grid in this side image.

I’m sure you can imagine how easy is to create a control class to deal with the actions in your 2D application and then create view classes referencing the control classes, so you can prepare the view classes to get the collision detection on many squares on the grid. So you have 1 action – N grid squares detectors. At this way you can take all advantages of the grid and also an incredible boost in your application’s performance.

By using the grid you can easy define the collision areas which are impossible to the character pass through, like the walls. Another great advantage of using the grid is define “top areas”, that means, areas which always will be drawn on top, like the up side of the trees. So if the character pass through these areas, he will be displayed behind.

The following image shows a final scene which uses all these concepts of the grid. Notice how many images can be overlapped by others, pay attention of how the character deals with the action square and its own top areas. And notice the most top effects overlapping everything, like the clouds shadows or the yellow light coming from the sun.

RPG Maker final scene.

Summarizing the points taken, the grid is really the most important part of planning a 2D application. The grid is not a real thing in OpenGL, so you have to be careful about using this concept, because everything will be imaginary. Well, just to let you know, an extra information: the grid concept is so important that the OpenGL internally works with a grid concept to construct the Fragments.

Great, this is everything about the grid. Now you would say: “OK, but this is not what I wanted, I want a game with a projection like Diablo, Sin City or even the We Rule does!”. Oh right, so let’s make things more complex and bring back the Depth Render Buffer and Cameras to our 2D application.

The Depth Render Buffer in 2D

top
Knowing how the 2D graphics goes with OpenGL, we can think in more refined approach, like use the Depth Buffer even in 2D applications.

"2D game" using OpenGL with Depth Render Buffer. (click to enlarge)

"2D game" using OpenGL without Depth Render Buffer. (click to enlarge)

By clicking on the images above, you can notice the difference between them. Both screenshot are from famous games from iOS, both use OpenGL and both are known as 2D games. Although they use OpenGL ES 1.1, we can understand the concept of Grid + Depth Render Buffer. The game on the left (Gun Bros) makes use of a very small grid, exactly 8×8 pixels, this kind of grid gives to the game a incredible precision to place the objects on the grid, but to improve the user experience you need to make a set of grid squares to deal with the actions, in this case a good choice could be arranging 4 ou 8 grid squares to each action detector. Now the game on the right, it’s called Inotia, by the way, Inotia is today in its 3th edition. Since the first edition, Inotia always used a big grid, 32×32 pixels. As Gun Bros, Inotia uses OpenGL ES 1.1.

There are many differences between those two grid types (8×8 and 32×32). The first one (8×8) is much more precisely and could seem to be the best choice, but remember that this choice will increase too much your processing. The Inotia game has a light processing demand, something absolutely unimpressive to the iOS’ hardwares. You need to make the best choice to fit within the application you are planning to use.

Now, talking about the Depth Render Buffer, the great thing about it is that you can use 3D models in your application. Look, without Depth Render Buffer you must use only squares, or another primitive geometric form, with textures. By doing this you must create one different texture to each position of your animation, specially to characters animation, obviously a great idea is make use of texture atlas:

Character texture atlas from Ragnarok.

The image of Inotia game above has a similar texture altas to each character that appear in the game. Looking at that image you can see that the three character on the screen can turn just to four directions. Now, take another look to the Gun Bros image above.

Notice that the characters can turn to all directions. Why? Well, by using a Depth Render Buffer you are free to use 3D models in your 2D application. So you can rotate, scale and translate the 3D models respecting the grid and the 2D concepts (no Z translate). The result is much much better, but as any improvements, it has a great cost for performance compared to 2D squares, of couse.

But there is another important thing about mixing 3D and 2D concepts: the use of cameras. Instead of creating a single plane right in front of the screen, locking the Z translations, you can create a great plane along the Z axis, place your objects just as in a 3D application and create a camera with orthographic projection. You remember what it is and how to do it, right? (click here to check the article about cameras).

Before going further into the cameras and Depth Render Buffer with 2D graphics, it’s important to know that at this point it has no real difference, at the code level, between 2D and 3D graphics since everything comes from your own planning and organization. So the code to use the Depth buffer is the same we saw in the last part (click here to see the last tutorial).

Now let’s talk about the cameras with 2D graphics.

The Cameras with 2D

top
OK, I’m sure you know how to create a camera and an orthographic projection now, as you’ve seen it at the tutorial about cameras, right? (click here to check the cameras tutorial). Now a question comes up: “Where is the best place and approach to use cameras and depth render buffer in the 2D graphics?”. The following image could help more than 1.000 words:

Same camera in both projections. (click to enlarge)

This image shows a scene like Diablo Game style, with a camera in the similar position. You can notice clearly the difference between both projections. Notice the red lines across the picture, with the Orthographic projection those lines are parallels, but with the Perspective projection those lines are not really parallels and can touch at the infinity.

Now focus on the grey scale picture at the bottom right. That is the scene with the objects. As you can see, they are really 3D objects, but with an orthographic projection you can create scenes like Diablo, Sim City, Starcraft and other best sellers, giving a 2D look to your 3D application.

If you take another look at that image of Gun Bros game, you can see that it’s exactly what they do, there is a camera with orthographic projection and real 3D objects placed on the scene.

So the best approach is to create a camera in your desired position, construct all your scene in a 3D world, set the camera to use orthographic projection and guide your space changes by using the grid concept.

The grid concept is very important even with cameras and Depth Render Buffer.

I have one last advice about this subject, well… it’s not really an advice, it’s more like a warning. Perspective and Orthographic projections are completely different. So the same configuration of focal point, angle of view, near and far produces completely different results. So you need to find a configuration to the Orthographic projection different of that you were using with Perspective projection. Probably if you have a perspective projection which is working, when you change to orthographic projection you won’t see anything. This is not a bug, it’s just the differences between perspective and orthographic calculations.

OK, these are the most important concepts about 2D graphics with OpenGL. Let’s make a little review of them.

• There are two ways of using 2D graphics with OpenGL: by using or not the Depth Render Buffer.
• Without Depth Render Buffer you can construct everything like a rectangle on the screen, forget the Z axis position at this way. Your job here will be laborious on textures. Although, by this way, you can have the best performance with OpenGL.
• By using the Depth Render Buffer you can use real 3D objects and probably you will want to use a camera with orthographic projection.
• Independent of the way you choose, always use the Grid concept when working on 2D graphics. It’s the best way to organize your world and optimize your application performance.

Now is time to go back into the 3D world and talk a little about the multisampling and antialias filter.

The Multisampling

top
I’m sure you already noticed that every 3D application, which have a real time render, have their objects’ edges aliased. I’m talking about 3D world in general, like 3D softwares or games, whatever… the edges always (I mean, in majority of the cases) looks like it’s kind aliased. That doesn’t happen due a lack of well developed techniques to fit that but rather it’s because our hardwares are not so powerful yet to deal with pixel blend in real time too fast.

So, the first thing I want to say about the Anti-Alias filter is: “it’s expensive!”. In the majority of cases this little problem (aliased edges) doesn’t matter. But there are some situations that your 3D application needs to looks better. The most simple and common example is the render in 3D softwares. When we hit the render button on our 3D software we expect to see gorgeous images, not jagged edges.

With simple words, the OpenGL primitives get rasterized onto a grid (yes, like our grid concept), and their edges may become deformed. OpenGL ES 2.0 supports something called multisampling. It’s a technique of Anti-Alias filter which each pixel is divided into few samples, each of these samples are treated like a mini-pixel in the rasterization process. Each sample has its own information about color, depth and stencil. When you ask the OpenGL for the final image on the Frame Buffer, it will resolve and mix all the samples. This process produces more smooth edges. OpenGL ES 2.0 is always configured to multisampling technique, even if the number of samples is equal 1, that means 1 pixel = 1 sample. Looks very simple in theory, but remember that OpenGL doesn’t know anything about the device’s surface, consequently, anything about device’s pixel and color.

The bridge between OpenGL and the device is made by the EGL. So the device color informations, pixel informations and surface informations are responsibility of EGL and consequentially the multisampling could not be implemented only by the OpenGL, it needs a plugin, which is responsibility of the vendors. Each vendors must create a plugin to EGL instructing about the necessary information, by doing this the OpenGL can really resolve the multi samples. The default EGL API offers a multisampling configuration, but commonly the vendors make some changes on it.

In the case of Apple, this plugin is called “Multisample APPLE” and it’s located at the OpenGL Extensions Header (glext.h). To correctly implement the Apple Multisample you need 2 Frame Buffer and 4 Render Buffer! 1 Frame Buffer is the normal provided by OpenGL, the another one is the Multisample Framebuffer. The Render Buffers are Color and Depth.

There are three new functions in the glext.h to deal with Multisample APPLE:

Before checking the code, let’s understand a little bit more about these new functions. The first function (glRenderbufferStorageMultisampleAPPLE) is intended to replace that function that set the properties to the Render Buffer, the glRenderbufferStorage. The big new in this function is the number of samples, it will define how many samples each pixel will has.

The second one (glResolveMultisampleFramebufferAPPLE) is used to take the informations from the original frame buffer, place it in the Multisample Frame Buffer, resolve the samples of each pixel and then draw the resulting image to our original Frame Buffer again. In simple words, this is the core of Multisample APPLE, this is the function which makes all the job.

The last one (glDiscardFramebufferEXT) is another clearing function. As you imagine, after the glResolveMultisampleFramebufferAPPLE makes all the processing, the Multisample Frame Buffer will be with many informations in it, so it’s time to clear all that memory. To do that, we call the glDiscardFramebufferEXT informing what we want to clear from where.

Now, here is the full code to use Multisample APPLE:

.
// EAGL
// Assume that _eaglLayer is a CAEAGLLayer data type and was already defined.
// Assume that _context is an EAGLContext data type and was already defined.

// Dimensions
int _width, _height;

// Normal Buffers
GLuint _frameBuffer, _colorBuffer, _depthBuffer;

// Multisample Buffers
GLuint _msaaFrameBuffer, _msaaColorBuffer, _msaaDepthBuffer;
int _sample = 4; // This represents the number of samples.

// Normal Frame Buffer
glGenFramebuffers(1, &_frameBuffer);
glBindFramebuffer(GL_FRAMEBUFFER, _frameBuffer);

// Normal Color Render Buffer
glGenRenderbuffers(1, &_colorBuffer);
glBindRenderbuffer(GL_RENDERBUFFER, _colorBuffer);
[_context renderbufferStorage:GL_RENDERBUFFER fromDrawable:_eaglLayer];
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, _colorBuffer);

// Retrieves the width and height to the EAGL Layer, just necessary if the width and height was not informed.
glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_WIDTH, & _width);
glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_HEIGHT, & _height);

// Normal Depth Render Buffer
glGenRenderbuffers(1, &_depthBuffer);
glBindRenderbuffer(GL_RENDERBUFFER, _depthBuffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16, _width, _height);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, _depthBuffer);
glEnable(GL_DEPTH_TEST);

// Multisample Frame Buffer
glGenFramebuffers(1, &_msaaFrameBuffer);
glBindFramebuffer(GL_FRAMEBUFFER, _msaaFrameBuffer);

// Multisample  Color Render Buffer
glGenRenderbuffers(1, &_msaaColorBuffer);
glBindRenderbuffer(GL_RENDERBUFFER, _msaaColorBuffer);
glRenderbufferStorageMultisampleAPPLE(GL_RENDERBUFFER, _samples, GL_RGBA8_OES, _width, _height);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, _msaaColorBuffer);

// Multisample Depth Render Buffer
glGenRenderbuffers(1, &_msaaDepthBuffer);
glBindRenderbuffer(GL_RENDERBUFFER, _msaaDepthBuffer);
glRenderbufferStorageMultisampleAPPLE(GL_RENDERBUFFER, _samples, GL_DEPTH_COMPONENT16, _width, _height);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, _msaaDepthBuffer);
.

Yes, many lines to a basic configuration. Once all those 6 buffers have been defined, we also need to make the render by a different approach. Here is the necessary code:

.
//-------------------------
//	Pre-Render
//-------------------------
// Clears normal Frame Buffer
glBindFramebuffer(GL_FRAMEBUFFER, _frameBuffer);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

// Clears multisample Frame Buffer
glBindFramebuffer(GL_FRAMEBUFFER, _msaaFrameBuffer);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

//-------------------------
//	Drawing
//-------------------------
//...
// Draw all your content.
//...

//-------------------------
//	Render
//-------------------------
// Resolving Multisample Frame Buffer.
glBindFramebuffer(GL_DRAW_FRAMEBUFFER_APPLE, _frameBuffer);
glBindFramebuffer(GL_READ_FRAMEBUFFER_APPLE, _msaaFrameBuffer);
glResolveMultisampleFramebufferAPPLE();

// Apple (and the khronos group) encourages you to discard
// render buffer contents whenever is possible.
GLenum attachments[] = {GL_COLOR_ATTACHMENT0, GL_DEPTH_ATTACHMENT};
glDiscardFramebufferEXT(GL_READ_FRAMEBUFFER_APPLE, 2, attachments);

// Presents the final result at the screen.
glBindRenderbuffer(GL_RENDERBUFFER, _colorBuffer);
[_context presentRenderbuffer:GL_RENDERBUFFER];
.

If you want to remember something about the EAGL (the EGL implementation by Apple), check it here: article about EGL and EAGL. OpenGL also offers some configurations to multisampling, glSampleCoverage and few configurations with glEnable. I’ll not talk in deep about these configurations here, because I don’t believe multisampling is a good thing to spend our time on. As I told you, the result is not a big deal, it’s just a little bit more refined. In my opinion, the performance cost is too much compared to the final result:

Same 3D model rendered without and with Anti-Alias filter.

OK, now it’s time to talk more about the textures in OpenGL.

More About Textures

top
We already know many things about textures from the second part of this serie (All About OpenGL ES 2.x – Textures). First, let’s talk about the optimized types. It’s a great boost on our application’s performance and is very easy to implement. I’m talking about the bytes per pixel of our images.

Bytes per Pixel

top
Usually the images has 4 bytes per pixel, one byte for each channel, RGBA. Some images without alpha, like JPG file format, has only 3 bytes per pixel (RGB). Each byte could be represented by an hexadecimal color in the format 0xFF, it’s called hexadecimal because each decimal house has the range 0 – F (0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F), so when you make a combination of two hexadecimal numbers you get one byte (16 x 16 = 256). As a convention, we describe a hexadecimal color as 0xFFFFFF, where each set of two number represent one color channel (RGB). For images with alpha channel, like PNG format, we are used to say 0xFFFFFF + 0xFF, that means (RGB + A).

My next article will be about binary programming, so I’ll not talk in deep about binaries here. All that we need for now is to know that 1 byte = 1 color channel. The OpenGL can also work with more compressed format, which uses only 2 bytes per pixel. What that means? It means that each byte will store two color channels, including alpha. In very simple words, let’s reduce the color’s range of the image.

The OpenGL offers to us 3 compressed data types: GL_UNSIGNED_SHORT_4_4_4_4, GL_UNSIGNED_SHORT_5_5_5_1, GL_UNSIGNED_SHORT_5_6_5. The first two should be used when you have alpha channel and the last one it’s only for situations without alpha. These 3 names tell us something about the pixel data. The numbers at the right instruct us about the number of bits (not bytes) that must be used in each channel (RGBA). Oh, and just to make it clear, each byte is composed by 8 bits. So in the first case 4 bits per channel with a total of 2 bytes per pixel. The second one, 5 bits to RGB channels and 1 bit to alpha with a total of 2 bytes per pixel. And the last one, 5 bits to R, 6 bits to G and 5 bits to B with a total of 2 bytes per pixel.

Here I want to make a warning: The type GL_UNSIGNED_SHORT_5_5_5_1 is not really useful, because only 1 bit to alpha is the same as give it a Boolean data type, I mean, 1 bit to alpha means visible YES or NOT, just it. So this type is not really useful, it has less bits on Green channel than GL_UNSIGNED_SHORT_5_6_5 and even can’t produce real transparent effects like GL_UNSIGNED_SHORT_4_4_4_4. So if you need the alpha channel, make use of GL_UNSIGNED_SHORT_4_4_4_4, if not, make use of GL_UNSIGNED_SHORT_5_6_5.

A little thing to know about the GL_UNSIGNED_SHORT_5_6_5. As the human eye has more sensibility to the green colors, the channel with more bits is exactly the Green channel. At this way, even with a less color range, the resulting image to the final user will not seem to be that different.

Now let’s take a look at the difference between both compressions.

OpenGL optimized 2 bytes per pixel (bpp) data types.

As you saw, by using GL_UNSIGNED_SHORT_4_4_4_4 could really be ugly in some situations. But the GL_UNSIGNED_SHORT_5_6_5 looks very nice. Why? I’ll explain in details at the next article about binaries, but in very simple words, by using GL_UNSIGNED_SHORT_4_4_4_4 we have only 16 tonalities for each channel, including 16 tonalities to alpha. But with GL_UNSIGNED_SHORT_5_6_5 we have 32 tonalities to Red and Blue and 96 tonalities of Green spectrum. It still far from the human eye’s capacity, but remember that by using these optimizations we reduce 2 bytes per pixel in all our images, this represents much more performance to our renders.

Now it’s time to learn how to convert our traditional images to these formats. Normally, when you extract the binary informations from an image you get it pixel by pixel, so probably each pixel will be composed by an “unsigned int” data type, which has 4 bytes. Each programming language provides a method(s) to extract the binary information from the pixels. Once you have your array of pixel data (array of unsigned int) you can use the following code to convert that data to the GL_UNSIGNED_SHORT_4_4_4_4 or GL_UNSIGNED_SHORT_5_6_5.

.
typedef enum
{
	ColorFormatRGB565,
	ColorFormatRGBA4444,
} ColorFormat;

static void optimizePixelData(ColorFormat color, int pixelDataLength, void *pixelData)
{
	int i;
	int length = pixelDataLength;

	void *newData;

	// Pointer to pixel information of 32 bits (R8 + G8 + B8 + A8).
	// 4 bytes per pixel.
	unsigned int *inPixel32;

	// Pointer to new pixel information of 16 bits (R5 + G6 + B5)
	// or (R4 + G4 + B4 + A4).
	// 2 bytes per pixel.
	unsigned short *outPixel16;

	newData = malloc(length * sizeof(unsigned short));
	inPixel32 = (unsigned int *)pixelData;
	outPixel16 = (unsigned short *)newData;

	if(color == ColorFormatRGB565)
	{
		// Using pointer arithmetic, move the pointer over the original data.
		for(i = 0; i < length; ++i, ++inPixel32)
		{
			// Makes the convertion, ignoring the alpha channel, as following:
			// 1 -	Isolates the Red channel, discards 3 bits (8 - 3), then pushes to the final position.
			// 2 -	Isolates the Green channel, discards 2 bits (8 - 2), then pushes to the final position.
			// 3 -	Isolates the Blue channel, discards 3 bits (8 - 3), then pushes to the final position.
			*outPixel16++ = (((( *inPixel32 >> 0 ) & 0xFF ) >> 3 ) << 11 ) |
							(((( *inPixel32 >> 8 ) & 0xFF ) >> 2 ) << 5 ) |
							(((( *inPixel32 >> 16 ) & 0xFF ) >> 3 ) << 0 );
		}
	}
	else if(color == ColorFormatRGBA4444)
	{
		// Using pointer arithmetic, move the pointer over the original data.
		for(i = 0; i < length; ++i, ++inPixel32)
		{
			// Makes the convertion, as following:
			// 1 -	Isolates the Red channel, discards 4 bits (8 - 4), then pushes to the final position.
			// 2 -	Isolates the Green channel, discards 4 bits (8 - 4), then pushes to the final position.
			// 3 -	Isolates the Blue channel, discards 4 bits (8 - 4), then pushes to the final position.
			// 4 -	Isolates the Alpha channel, discards 4 bits (8 - 4), then pushes to the final position.
			*outPixel16++ = (((( *inPixel32 >> 0 ) & 0xFF ) >> 4 ) << 12 ) |
							(((( *inPixel32 >> 8 ) & 0xFF ) >> 4 ) << 8 ) |
							(((( *inPixel32 >> 16 ) & 0xFF ) >> 4 ) << 4 ) |
							(((( *inPixel32 >> 24 ) & 0xFF ) >> 4 ) << 0 );
		}
	}

	free(pixelData);

	pixelData = newData;
}
.

The routine above assumes the channel order as RGBA. Although is not common, your image could have the pixels composed by another channel order, like ARGB or BGR. In these cases you must change the routine above or change the channel order at when extracting binary informations from each pixel. Another important thing is about the binary order. I don't want to confuse your mind if you don't know too much about binaries, but just as an advice: you probably will get the pixel data in little endian format, the traditional, but if your programming language get the binary informations as a big endian, the above routine will not work properly, so make sure your pixel data is in little endian format.

PVRTC

top
I sure you've heard about the texture compression format PVRTC, if you already feel confortable about this topic, just skip to the next one. PVRTC is a binary format created by "Imagination Technology", also called "Imgtec". This format uses the channel order as ARGB instead the traditional RGBA. To say the truth, its optimization is not about the file's size, if we look only to the size, any JPG is more compressed or even a PNG can be lighter. The PVRTC is optimized about its processing, its pixels can be already in the format 2 bytes per pixel (2bpp) or 4 bytes per pixel (4bpp). The data inside PVRTC is OpenGL friendly and also can store Mipmap levels. So, could it be a good idea to always make use of PVRTC? Well, not exactly... let's see why.

The PVRTC format is not supported by default in OpenGL ES 2.0, there are informations that OpenGL ES 2.1 will come with native support to PVRTC textures, but what we have for now is just the OpenGL ES 2.0. To use PVRTC on it, just as the Multisampling, you need a vendors plug-in. In the case of Apple, this plugin has four new constant values. OpenGL provides a function to upload pixel data from the compressed formats, like PVRTC:

As you can imagine, the data format GL_UNSIGNED_SHORT_4_4_4_4 or GL_UNSIGNED_SHORT_5_6_5 is chosen based on the file format, RGB or RGBA with 2bpp or 4bpp, depends on.

To generate the PVRTC you have many options. The two most common is the Imgtec Tools or the Apple's Texture Tool. Here you can find the Imgtec Tools. The Apple tool comes with iPhone SDK, it's located at the path "/iPhoneOS.platform/Developer/usr/bin" the name is "texturetool", you can find all informations about it at Apple Texture Tool.

I'll explain how to use the Apple tool here. Follow these steps:

• Open the Terminal.app (usually it is in /Applications/Utilities/Terminal.app)
• Click on texturetool in Finder and drag & drop it on Terminal window. Well, you also can write the full path "/iPhoneOS.platform/Developer/usr/bin/texturetool", I preffer drag & drop.
• Write in front of texture tool path: " -e PVRTC --channel-weighting-linear --bits-per-pixel-2 -o "
• Now you should write the output path, again, I prefer drag & drop the file from the Finder to the Terminal window and rename its extension. The extension really doesn't matter, but my advice is to write something that let you identify the file format, like pvrl2 for Channel Weighting Linear with 2bpp.
• Finally, add an space and write the input file. Guess... I prefer to drag & drop from the Finder. The input files must be PNG or JPG files only.
• Hit "Enter"

.
/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/texturetool
 -e PVRTC --channel-weighting-linear --bits-per-pixel-2 -o
/Texture/Output/Path/Texture.pvrl2 /Texture/Intput/Path/Texture.jpg
.

Good, now you have a PVRTC file. The problem with the Apple tool is that it doesn't generate the traditional PVRTC binary header. It's composed by 52 bytes at the beginning of the file and gives instructions about the height and width of the image, number of Mipmaps on it, bpp, channels order, alpha, etc. In the traditional PVRTC files, this is the header format:

• unsigned int (4 bytes): Header Length in Bytes. Old PVRTC has a header of 44 bytes instead 52.
• unsigned int (4 bytes): Height of the image. PVRTC only accepts squared images (width = height) and POT sizes (Power of Two)
• unsigned int (4 bytes): Width of the image. PVRTC only accepts squared images (width = height) and POT sizes (Power of Two).
• unsigned int (4 bytes): Number of Mipmaps.
• unsigned int (4 bytes): Flags.
• unsigned int (4 bytes): Data Length of the image.
• unsigned int (4 bytes): The bpp.
• unsigned int (4 bytes): Bitmask Red.
• unsigned int (4 bytes): Bitmask Green.
• unsigned int (4 bytes): Bitmask Blue.
• unsigned int (4 bytes): Bitmask Alpha.
• unsigned int (4 bytes): The PVR Tag.
• unsigned int (4 bytes): Number of Surfaces.

But, using the Apple Texture Tool we don't have the file header and without that header we can't find neither the width nor height of the file from our code. So to use PVRTC from Apple tool you should know about bpp, width, height and alpha. Kind of annoying, no?

Well... I have good news for you. I found a way, a trick, to extract informations from the PVRTC generated by Apple tool. This trick works fine, but it can't identify informations about the Mipmap, but this is not a problem, because Apple tool doesn't generate Mipmaps anyway.

.
// Supposing the bpp of the image is 4, calculates its squared size.
float size = sqrtf([data length] * 8 / 4);

// Checks if the bpp is really 4 by comparing the rest of division by 8,
// the minimum size of PVRTC, if the rest is zero then this image really
// has 4 bpp, otherwise, it has 2 bpp.
bpp = ((int)size % 8 == 0) ? 4 : 2;

// Knowing the bpp, calculates the width and height
// based on the data size.
width = sqrtf([data length] * 8 / bpp);
height = sqrtf([data length] * 8 / bpp);
length = [data length];
.

The PVRTC files made from Texturetool doesn't have any header, so its image data starts in the first byte of the file. And what about the alpha? Could you ask. Well, the alpha will be more dependent of your EAGL context configuration. If you are using RGBA8, assume the alpha exist and use the GL_COMPRESSED_RGBA_PVRTC_4BPPV1_IMG or GL_COMPRESSED_RGBA_PVRTC_2BPPV1_IMG, based on the informations that you extract from the code above. If your EAGL context uses RGB565, so assume GL_COMPRESSED_RGB_PVRTC_4BPPV1_IMG or GL_COMPRESSED_RGB_PVRTC_2BPPV1_IMG.

Now to use your PVRTC on OpenGL ES 2.0, it's very simple, you don't need to change almost anything, you will create your texture normally, just replace the call to glTexImage2D by glCompressedTexImage2D function.

.
// format = one of the GL_COMPRESSED_RGB* constants.
// width = width extract from the code above.
// height = height extract from the code above.
// length = length extract from the code above.
// data = the array of pixel data loaded via NSData or any other binary class.

// You probably will use NSData to load the PVRTC file.
// By using "dataWithContentsOfFile" or similar NSData methods.

glCompressedTexImage2D(GL_TEXTURE_2D, 0, format, width, height, 0, length, data);
.

Well done, this is all about PVRTC. But my last advice about this topic is, always avoid to use PVRTC. The cost X benefit is not so good. Remember you just need to parse an image file once to OpenGL, so PVRTC doesn't offers a great optimization.

The Off-Screen Render

Until now we've just talked about render to the screen, "on the screen", "on the device", but we also have another surface to render, the off-screen surfaces. You remember it from the EGL article, right? (EGL and EAGL article).

What is the utility of an off-screen render? We can take a snapshot from the current frame and save it as an image file, but the most important thing about off-screen renders is to create an OpenGL texture with the current frame and then use this new internal texture to make a reflection map, a real-time reflection. I'll not talk about the reflections here, this subject is more appropriated to a tutorial specific about the shaders and lights, let's focus only about how to render to an off-screen surface. We'll need to know a new function:

To use this function we need to first create the target texture. We can do it just as before (check out the texture functions here). Then we call glFramebufferTexture2D and proceed normally with our render routine. After drawing something (glDraw* callings) that texture object will be filled and you can use it for anything you want. Here is an example:

.
// Create and bind the Frame Buffer.
// Create and attach the Render Buffers, except the render buffer which will
// receive the texture as attachment.

GLuint _texture;
glGenTextures(1, &_texture);
glBindTexture(GL_TEXTURE_2D, _texture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA,  textureWidth, textureHeight, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, _texture, 0);
.

Differently than creating a texture for a pixel data, at this time you set the texture data to NULL. Because they will be filled up dynamically later on. If you intend to use the output image as a texture for another draw, remember first to draw the objects that will fill the output texture.

Well, as any Frame Buffer operation, it's a good idea to check glCheckFramebufferStatus to see if everything was attached OK. A new question comes up: "If I want to save the resulting texture to a file, how could I retrieve the pixel data from the texture?", OpenGL is a good mother, she gives us this function:

As you've saw, the function is very easy, you can call it any time you want. Just remember a very important thing: "OpenGL Pixel Order!", it starts in the lower left corner and goes to the up right corner. To a traditional image file, that means the image is flipped vertically, so if you want to save to a file, take care with it.

Now you must to do the invert path you are used to when import a texture. Now you have the pixel data and want to construct a file. Fortunately many languages offers a simple way to construct an imagem from pixel data. For example, with Cocoa Touch (Objective-C), we can use NSData + UIImage, at this way:

.
// The pixelData variable is a "void *" initialized with memory allocated.

glReadPixels (0, 0, 256, 256, GL_RGB, GL_UNSIGNED_BYTE, pixelData);
UIImage *image = [UIImage imageWithData:[NSData dataWithBytes:pixelData length:256 * 256]];

// Now you can save the image as JPG or PNG.
[UIImageJPEGRepresentation(image, 100) writeToFile:@"A path to save the file" atomically:YES];
.

Just a little question, glReadPixels will read from where? OpenGL State Machine, do you remember? glReadPixels will read the pixels from the "last Frame Buffer bound".

Now it's time to talk more about optimization.

Tips and Tricks

top
I want to talk now about some tips and tricks to boost your application. I don't want to talk about little optimizations which make you gain 0.001 secs, no. I want to talk about real optimizations. That ones which can boost 0.5 secs or even increase your render frame rate.

The Cache

top
This is very important, I really love it, I'm used to use it on everything, it's great! Imagine this situation, the user touch the objects on the screen to make a rotation. Now the user touch another object, but the first doesn't change anything. So it would be great to make the first object's transformation matrix a cached matrix, instead to recalculating the first object's matrix at each frame.

The cached concept extends even to other areas, like cameras, lights and quaternions. Instead to recalculate something at each frame, use a little BOOL data type to check if a matrix or even a value is cached or not. The following pseudo-code shows how is simple to work with cache concept.

.
bool _matrixCached;
float _changeValue;
float *_matrix;

float *matrix(void)
{
    if (!_matrixCached)
    {
        // Do changes into _matrix.

        _matrixCached = TRUE;
    }

    return _matrix;
}

void setChange(float value)
{
    // Change the _changeValue which will affect the matrix.

    _matrixCached = FALSE;
}
.

Store the Values

top
We are used to change the matrix (or the quaternion) every time that occurs transformation. For example, if our code make the changes: translate X - we change the resulting matrix, translate Y - we change the matrix, rotate Z - change the matrix and scale Y - change the matrix. Some 3D engines and people do not even hold on a value to those transformations, so if the code need to retrieve those values, they extract the values directly from the resulting matrix. But this is not the best approach. A great optimization could be reached if we store the values independently, like translations X,Y and Z, rotations X,Y and Z and scales X,Y and Z.

By storing the values you can make a single change into the resulting matrix, making the calculations once per frame instead of making the calculus at every transformation. The following pseudo-code can help you to understand the Store concept better:

.
float _x;
float _y;
float _z;

float x(void) { return _x; }
float setX(float value)
{
    _x = value;
}

float y(void) { return _y; }
float setY(float value)
{
    _y = value;
}

float z(void) { return _z; }
float setZ(float value)
{
    _z = value;
}

float *matrix(void)
{
    // This function will be called once per frame.
    // Make the changes to the matrix based on _x, _y and _z.
}
.

C is always the fast language

top
This tip is just something to remember. You probably know that, but it's very important to reinforce. C is always the fastest language. No other language can be faster than C. It's the most basic language and it's the great father of almost all computer languages. So always try to use C in the most critical parts of the code. Specially for render routines.

String comparisons with C is around 4x faster than Objective-C comparisons. So if you need to check some string value at the render time, prefer to convert it from NSString to C string (char *) and make the comparison, even if you need re-convert from C string to NSString again, even in these cases C string is faster. To compare C strings you know, just use if (strcmp(string1, string2) == 0).

Specially for numbers, always use basic C data types (float, int, short, char and their unsigned versions). Besides, avoid at maximum value that use 64 bits, like long or double data type. Remember that OpenGL ES doesn't support, by default, 64 bits data types.

Conclusion

top
OK dudes, we're at the end of our objective with this serie. I'm sure now you know a lot of things about OpenGL and 3D world. I have covered almost all about the OpenGL in those 3 tutorials of this serie. I hope you learned all the concepts about the subjects covered in these tutorials.

Now, as we are used, let's remember everything:

• 2D graphics with OpenGL can be done by two ways: with or without Depth Render Buffer.
• When using Depth Render Buffer, it could also be good to make use of a camera with Orthographic projection.
• Independent of the way you choose, always use the Grid concept with 2D graphics.
• The Multisampling filter is a plug-in that depends the vendors implementation. Multisampling always has a big cost on performance, use it only in special situations.
• Always try to optimize your textures to a 2bpp data format.
• You can use PVRTC in your application to save some time when creating an OpenGL texture from a file.
• Always try to use the Cache concept when working with matrices.
• Make use of the Store Values concept to save CPU processing.
• Prefer basic C language on the critical render routines.

Well, you know, if you have some doubt, just ask me, let a comment bellow and if I can help I'll be glad.

From here and beyond

Well, and now? Is this all? No. It will never be enough! The points and lines deserve a special article. With points in can make particles and some cool effects. As I told at the beginning of this tutorial, you can use points with 2D graphics instead squares, in case of no Depth Render Buffer.

And about the shaders? In deep? The Programmable Pipeline gives us a whole new world of programming. We should talk about the Surface Normals VS Vertex Normals, about the tangent space, the normal bump effect, the reflection and refraction effect. To say the truth... I think we need a new serie of tutorial called "All About OpenGL Shaders". Well, this could be my next serie.

But I want to hear from you, tell me here or on Twitter what you want to know more about.
Just Tweet me:
Tweet

Thanks again for reading.
See you in the next tutorial!

I felt a good impulse to write a little article about this theme, because there are few poor informations about it around the web. In many cases, are wrong or incomplete informations. So if you want to write a framework to iPhone and distribute it, you are in the right place!

I’ll treat here about how to construct an Universal Framework to iOS, what are the necessary configurations and everything else related to. We’ll focus on Xcode 4, but this is also valid to Xcode 3.x.

Let’s start!

–WARNING–

This content is outdated! There is a new version of this article (2.0) teaching how to create an Universal Framework to iOS, much simpler and reliable:
http://db-in.com/blog/2011/07/universal-framework-iphone-ios-2-0/

Here is a little list of contents to orient your reading:

You can download the project I used in this article:

Download now
Xcode project files to Framework-iOS
403kb

Framework on iOS? Really?

top
Ok buddies, let’s make something clear, many people had said:”iOS doesn’t support custom Frameworks!”, “Custom Framework is not allowed at iOS!”, “Doesn’t exist custom Framework on iOS!” and many other discouraging things like these. Look, I’ve made many frameworks and worked with many others, I don’t believe that is really impossible to use a Framework on iOS. According to my experience and knowledge about Frameworks, it’s absolutely feasible a custom Framework on iOS Devices. If we think more about the issue we can find an elegant solution, right? First, let’s understand what a Framework really is, here is the definition of framework by Apple’s eyes:

A framework is a hierarchical directory that encapsulates shared resources, such as a dynamic shared library, nib files, image files, localized strings, header files, and reference documentation in a single package.

Doesn’t make many sense something with this description not be allowed in iOS, thinking in architecture and structure. Apple also says:

A framework is also a bundle and its contents can be accessed using Core Foundation Bundle Services or the Cocoa NSBundle class. However, unlike most bundles, a framework bundle does not appear in the Finder as an opaque file. A framework bundle is a standard directory that the user can navigate.

Good, now thinking about iOS security, performance and size, the only thing in a Framework definition which doesn’t fit in iOS technology is the “dynamic shared library”. The words “dynamic” and “shared” are not welcome in the iOS architecture. So the Apple allows us to work and distribute something called “Static Library“. I don’t like it! It’s not so easy as a Cocoa Framework, if a developer got your Static Library, he needs to set a header path too, or import the header files… it’s not useful, that’s a shit!

Well, so a Framework concept is absolutely compatible with iOS, except by the “dynamic shared library”, on the other hand Apple says that a “static library” is OK for iOS. So if we replace the “dynamic shared libraries” by a “static library” we could construct a Custom Framework to iOS, right?

Right!!!!

This is exactly what we’ll make in this article, let’s construct a Framework with Static Library, by the way, an Universal Framework.

Understanding Universal, Dynamic and Static concepts

top
Simple answer:

• Universal: Which works perfect on all architectures. iOS devices uses armv6 and armv7, iOS simulator on MacOS X uses i386.
• Dynamic: The compiler doesn’t include the target files directly. The classes/libraries are already pre-compiled (binary format) and lies on the system path. Besides, the dynamic libraries can be shared by many applications. This is exactly what Cocoa Framework is.
• Static: It represents that classes/libraries which is compiled by the compiler at the build phase. These files can’t be shared by other applications and relies on application path.

Simple as that. If you need more informations about Dynamic VS Static libraries, try this Apple’s Documentation.

No more concepts, hands at work!

Constructing a Framework Project

top

1. Create the Project:

top
I want to show you step by step of the entire process, so let’s start with the most basic, create an iOS project. You can choose one application template in Xcode, this is not really so important, but remember to choose one template which could test your Framework code before export it.

Create an application project.

2. Framework Classes:

top

Create your framework classes.

Remember to create an import header to make everything more simple and organized to the user of your framework. Remember to write this header file with a framework notation, just as shown in the image bellow. Also remember to create your classes taking care to hide the classes which should not be visible to the other developers (users of your framework). We will set the public and private headers soon, but it’s very important to you protect the “core” classes, I mean, that classes which you don’t want to make visible to other developers.

For those private classes, you could import their header inside the “.m” (or .mm, .cpp, etc) file of a public class, by doing this you protect the header of private classes. Well, I know you probably already know that, I’m saying just to reinforce.

After you have all the classes (and also other files, like images, sounds, etc.), you are ready to compile a custom framework. In the next step, we’ll create a target to compile our framework, this will be our first big trick!

Create your framework header with framework notation.

Compiling a Framework

top

3. Create a Framework Target:

top
OK, let’s create a target to compile our framework. Click on the icon of your project in the project navigator at the left and hit the button “Add Target”. A new window will come up. Now is our first trick. Instead to create a “Cocoa Touch Static Library” or a “Cocoa Framework” we will create a “Bundle” target.

A Bundle? Really? Yes! I can explain. A “Cocoa Framework” target can’t be compiled to armv6/armv7 and Xcode doesn’t allow us to use “Static Libraries” in a “Cocoa Framework”, so we can’t use this target. On the other hand, we can’t use “Cocoa Touch Static Library” either, because it doesn’t use the framework structure that we want.

Now, the Bundle target could be the best choice. It can hold any file we want, we can compile source code inside it and… we can turn it into a framework. To say the truth, almost all “Framework & Library” targets could be turned into a framework too, even the “Cocoa Touch Static Library”, throughout this article you probably will figure out why. For now, let’s create a Bundle target.

Create a Bundle target rather than Cocoa Touch Static Library.

4. Bundle Setup:

top
It’s time to make all the necessary changes to the Bundle target. First of all, make sure you have cleaned up all the default files from the Bundle target. Remove the linked frameworks (you can find it clicking in the “Build Phase” tab and then they are in the section “Link Binary With Libraries“) and delete the Preference List file (.plist), the pre-compiled headers (.pch) and the language files.

I’m sure you already know this, but just to reinforce, here is the Build Setting screen, you can find it by clicking on the project icon in the left project navigator and then clicking in the “Build Setting” tab.

You must make a special Build Setting to turn a Bundle into a framework.

Here is our second great trick, or should be better to say “tricks”. Let’s change the “Build Setting” following this list:

• Architectures: Standard (armv6 armv7) (the values for this property depend on the value of the item bellow).
• Base SDK: Latest iOS (iOS X.X) (in the X.X will appear the number of the lastest iOS SDK installed on your machine).
• Build Active Architecture Only: NO (this is a very important set, if it’s YES then we can’t compile to armv6 and armv7 at the same time).
• Supported Platforms: iphonesimulator iphoneos.
• Valid Architecture: $(ARCHS_STANDARD_32_BIT) (it’s very important to be exactly this value, seems there is a bug in Xcode 4, once you set i386 as supported platforms to iOS, it can’t be removed anymore and it could generate errors in your project. So, to avoid any architectures error, use this value).
• Installation Directory: [optional change] I like to set this to $(BUILT_PRODUCTS_DIR), but this is not really relevant to our purposes here. You can set it to a path of your choice.
• Mac OS X Deployment Target: Compiler Default.
• Dead Code Stripping: NO.
• Link With Standard Libraries: NO.
• Mach-O Type: Relocatable Object File. This is the most important change. Here, we instruct the compiler to treat the Bundle as a relocatable file, by doing this, we can turn it into a framework with the wrapper setting.
• Other Linker Flags: [optional change] -ObjC, could be good make sure the compiler understands what’s the language it is compiling for, this can reduce the warnings at the compilation.
• Info.plist File: empty, remove any value from this field.
• Wrapper Extension: framework. Here we change the Bundle to a Framework. To Xcode, frameworks is just a folder with the extension .framework, which has inside one or more compiled binary sources, resources and some folders, a folder, usually called Headers, contains all the public headers.
• Precompile Prefix Header: NO.
• Prefix Header: empty, remove any value from this field.
• Generate Debug Symbols: NO (this is a very important setting, otherwise your framework will not work on another computer/profile).

5. Adding code and resources to the Bundle (Framework)

top
It’s time to place the content in our framework and define the public headers. To do that, with the Bundle target selected, click on the “Build Phase” tab. At bottom, hit the button “Add Phase” and then “Add Copy Headers“.

Open the recently created “Copy Headers” section and separate your public headers from the private or project headers. The difference here is:

• Public: Headers that other developers must know in order to work with your framework. In the final framework product, these headers will be visible even to Xcode.
• Private: Headers that is not necessary to other developers, but is good for consult or for reference. These headers will not be visible to Xcode, but will be in the framework folder.
• Project: Headers that the other developers nor Xcode have access. In reality these headers will not be placed in the final product, this is just to instruct the compiler to create your custom framework.

Now, open the “Compile Source” section and put there all your .m, .c, .mm, .cpp or any other compilable source file. If your framework include not compilable files, like images, sounds and other resources, you can place them in the “Copy Bundle Resources” section, in the example of this article, I used no resources. This is how your “Build Phase” will looks like:

Define your compilable source and the headers.

6. Schemes Setup

top
This is an important change to compile our framework. You don’t need to compile it in the Debug modes, we just need the Release mode, because if you have any Macro in your code which works only in Debug mode, like DEBUG macro, it will assume the configuration of the product that the other developers will make. So to us now, only the Release is important.

You can change it accessing the menu Product > Manage Schemes… or by the schemes short cut: the drop list in front at the Build/Run bottom in the top left corner of the Xcode window.

At this point could be a good idea to delete all the current schemes and then press the button “AutoCreate Schemes Now“. The important is select your framework target and hit the button “Edit” at the bottom. In the next window, change the “Build Configuration” from Debug to Release. Change to Release in all the situations: Run, Test, Profile, Analyze and Archive. By doing this, we ensure that our framework target will always compile the Release version.

Build the custom framework only to Release.

7. Building the Framework

top
OK, here is the annoying step of this process. Until now, I don’t figure out a more easy an elegant way to do this. You must to compile this target twice: one to iOS Device (this will compile for the architectures armv6/arvm7) and to iOS Simulator (can be iPhone or iPad with any SDK, doesn’t matter, the simulator always will compile for the architecture i386).

I’ve tried another way, like create an “integrator” and I placed two targets to its “Target Dependencies”, but trust me, it’s worst, because you need to change 2 targets instead 1 in cases when you change the framework content. Besides, the dependencies will assume the architectures of the “integrator”, so you need to compile twice too. Anyway… the best solution I found until now is use only 1 target and build it twice. If you find a better one, please tell me.

Compile the framework target twice: iOS Device and Simulator.

After the compile complete, you can see in the “Products” folder in the Project Navigator at the left, that your framework product now is active. Right click on it and hit “Show in Finder“. Take a look at this product, its is your iOS Framework! Great!

But we’re not done yet. You can test this product in other applications if you want, but it will only work to one architecture. We must to create an Universal version of this product. If you look around your framework product in the Finder, you’ll see that it is in a folder called “Release-iphoneos” (if you didn’t change the destination folder in the Build Settings). And if you have compiled to Simulator too, you’ll see another folder called “Release-iphonesimulator” which contains the same framework product, but this version is for architecture i386.

So, let’s join both products into one.

Building the Universal Framework

top

8. Creating Universal Target:

top
To join both architectures products into one, we must to use the Lipo Tool. It’s a tool which comes with iOS SDK, just to know, it is in “/Platforms/iPhoneOS.platform/Developer/usr/bin”, its file name is “lipo”. But we don’t need to know this path, Xcode can deal with it to us.

To use the Lipo Tool we’ll need to create a “Run Script” at the “Build Phase“, you can create it in your previously Bundle Target, but my advice is to create another target. I say to create another target to avoid compiling errors. This script will need to use “Release-iphoneos” and “Release-iphonesimulator” folders, so if the folders or products inside them not exist yet, the compiler will generate errors.

Let’s add a new target, hit the “Add Target” button, just as you did with Bundle Target. At this time a good choice is the “Aggregate” target. It doesn’t create any product directly, its purposes is just to aggregate another targets and run some scripts, exactly what we want!

Use the "Aggregate" target to construct a run script.

9. Lipo Tool Script:

top
We are almost there, now it’s time to create the Run Script to use the Lipo Tool. Well, you probably will need to make some changes in the code bellow to fit it to your framework name or folders (if you set different folders to build for). Here is the script to use:

.
if [ "${ACTION}" = "build" ]
then
# Set the target folders and the final framework product.
    INSTALL_DIR=${SYMROOT}/FI.framework
    DEVICE_DIR=${SYMROOT}/Release-iphoneos
    SIMULATOR_DIR=${SYMROOT}/Release-iphonesimulator

# Create and renews the final product folder.
    mkdir -p "${INSTALL_DIR}"
    rm -rf "${INSTALL_DIR}"
    rm -rf "${INSTALL_DIR}/FI"

# Copy the header files to the final product folder.
    ditto "${DEVICE_DIR}/FI.framework/Headers" "${INSTALL_DIR}/Headers"

# Use the Lipo Tool to merge both binary files (i386 + armv6/armv7) into one Universal final product.
    lipo -create "${DEVICE_DIR}/FI.framework/FI" "${SIMULATOR_DIR}/FI.framework/FI" -output "${INSTALL_DIR}/FI"
fi
.

Lipo script to join products for architectures i386 and armv6/armv7.

Now build the Aggregate target. At this time doesn’t matter if you build for iOS Device or Simulator, because all that it will make is run the script, nothing more. The above script creates the final Universal Framework at the ${SYMROOT} path, this is the same path of your Bundle target. So you can right click on the Framework product in the “Products” again and hit “Show in Finder”. Move to the level with “Release-iphoneos” and “Release-iphonesimulator” folders. Now, there is a new folder with the “.framework” extension, it’s your Universal Framework to iOS!

Congratulations!

Testing your Universal Framework

top

10. Testing:

top
To test your Universal Framework could be a good idea take it from that folder and place it in another one more easy to find, like the Desktop or ~/Library/Frameworks. Well, to avoid take it from that folder every time, you can change the Run Script above. Just change the “INSTALL_DIR” variable and set a path of your choice, make sure you choose a path with the write permissions. A good choice could be use the Xcode variable ${SRCROOT} which represents your project folder root, so you can set ${SRCROOT}/Products or something like that.

Anyway, with your Universal Framework in an “user friendly path”, create a new Xcode project, select the Application target and go to “Build Phase” tab. Open the section “Link Binary With Libraries” and hit the “+” to add a new Framework. Click the “Add Other…” button and select your Universal Framework. Remember, you must to select the “.framework” folder.

After the importing, Xcode will know everything about your custom framework. So try to write some code using your framework classes. Remember to import your Framework Principal Header as a framework notation. Xcode will use your public headers in the Code Completion.

Xcode knows your public headers and can use them in the Code Completion.

I like to use the Xcode Midnight Color Theme, so the classes of the project are green and the framework classes are blue. I love blue classes!

Conclusion

top
Well done, my friends! As we are used, let’s make a final review and take care with some possible problems.

• In a common Xcode project, create a Bundle target.
• Make the necessary setup, place your sources, headers and resources in it and finally change the its Schemes configuration.
• Compile to iOS Device (armv6/armv7) and to Simulator (i386).
• Create an Aggreate target and place a Run Script in it.
• Enjoy your real Universal Framework to iOS!

To finish I want to talk about some issues that I had:

• Take special care with the architectures. Don’t write the i386 directly in the Valid Architectures setting, always use the $(ARCHS_STANDARD_32_BIT). Plus, always set the Build Active Architecture Only to NO.
• Take care with your classes structure. If you set, for example, the ClassB.h as a Project or Private header, but in your code you import it into a Public header, this will cause conflicts.
• Take care with the Install Directory and Skip Install settings. They can generate warning if the folders permissions or paths are not right set.
• Finally, take a special care with the Lipo Tool script. It could generate warnings and/or errors if the paths was not found or the folder permissions was not right set.

This is all, buddy.
As I told you, the annoying thing is that you need build 3 times to generate a new Universal Framework: 2 x Bundle target (one to iOS Device and other to the Simulator) and 1x to the Aggregate target (which will run a script). But by my experience, this is not to much, you will create the final product only few times. At the great majority, you will test your Framework classes directly with the Application target in your Framework Xcode Project.

Enjoy your real Framework to iOS!

Thanks for reading,
See you soon!

Today I want to talk about the NinevehGL. Talk more about the features and about what it can offer to us. NinevehGL is almost done, I want to be as fast as possible, but I don’t want to launch it without everything seems great: the documentation, the tutorials, the official website and obviously, the NinevehGL itself.

Soon I’ll start posting videos and many divulgations about it, but right now in this article let’s see some images taken directly from the NinevehGL running.

Let’s start!

At a glance

The NinevehGL is a 3D engine fully made with the most pure Objective-C, right on top of Cocoa Touch Framework. So at the first moment, the NinevehGL is a 3D engine only for iOS. I intend to port NinevehGL to MacOSX (desktops with Obj-C) and also for ActionScript 3.0, using the new great API code named Molehill. ActionScript is my old programming language, today I don’t hold much lover for it, but I know that there are many people that still loving Flash. Another important step is to port NinevehGL to JavaScript to work with WebGL. But at the beginning, NinevehGL will work only with OpenGL ES 2.x for iOS.

Now the most important question: “Why NinevehGL instead of PowerVR, Oolong, SIO2, Torque, Ogre, UDK, Unity, Wolfenstein, Shiva, Galaxy, or any other? Why should I choose NinevehGL instead any other 3D engine?”.

I always tried to convince everyone to buy what I was selling, trying to convince everybody from my point of view. But not this time, I don’t want to sell anything, I don’t want to convince you. If you are using some other engine and glad with that, just continue with it, you don’t have to change. Also, if you are a great OpenGL programmer and have your very own model and structure, continue with it, don’t change anything. Because, again, I don’t want to convince anyone at this time. Just as OpenGL, I’ll focus on the most important thing to me: development!

I took everything that I’ve seen of worst in other engines and I’ve tried to make it better in NinevehGL. I took all my frustations with the actual engines, all my luggage as developer, all my knowledge about 3D world and put all of them in the NinevehGL. So, if you are like me and you have some insatisfaction with the other engines, for you, NinevehGL could really nice for you! I know I’m telling this every time, but I love this concept:”Keep it simple!” This is the NinevehGL’s main principle, everything was made to be simple. There are no commands that need more than one single line of code. If you want to make one thing, you need one line, nothing else.

Now I’ll describe in details the most important features of NinevehGL. Remember, I don’t want to convince you of anything. I’m sure you are an advanced programmer and can form your own opinion about what’s coming next. If you like it, great! If don’t, it’s great as well! Just tell me why and I’ll try to make better.

Here is a features’ list.

Importing 3D files

This is the most problematic topic to me considering other engines. As OpenGL doesn’t import 3D files directly, you must use an engine that imports a file good for you or use one of the files accepted by that engine, like POD to PowerVR engine.

This is the first problem that NinevehGL works to solve, it can import the two most popular 3D file formats: COLLADA and WaveFront OBJ. You know what a mess the 3D imports with 3D softwares are. Each software has its own file format and implements its importing routines at its own way. So it’s not hard to export an OBJ from 3DS Max and when you try to import in Maya, BOOM, error! To tell the truth, the incompatibility with the files when exporting from a 3D software to another is common.

The NinevehGL has a full importing routine. What that means? If your OBJ file has informations about the specular, the bump map, the reflection map or anything supported by OBJ file, NinevehGL will import and parse everything for you with no losses. The same is true for COLLADA files.

NinevehGL supports OBJ and DAE files.

Caching 3D files

This is the greatest feature in my opinion. NinevehGL works with a third kind of file, the NGL binary file. It’s a binary file containing a full 3D model with all informations necessary to NinevehGL and also OpenGL. This file is loaded through streaming, so it’s extremely fast. There are no 3D softwares that can export it, it’s exclusive from NinevehGL. So how you can use it? There are two ways: First, you can convert your OBJ or COLLADA file on-line! Yes, you read it right, on-line convertion! No plugins, no installations, no complex things, you can generate NGL file on-line! Just access the NinevehGL website, choose the files on your machine and hit the button!

The other way, is the most fascinating one. NGL file is generated automatically by NinevehGL at the first time you load a new file from OBJ or COLLADA. This file will be stored locally in the device under the folder /Library/NinevehGL. What is great here, is that this folder is fully backed up by iTunes. So imagine this:

1. You make an APP, which loads OBJ and COLLADA files.
2. An user make the download of your APP from the APP Store.
3. The user runs your APP at the first time, the NinevehGL loads around 20 3D file in 12 secs, let’s suppose.
4. The next times that this user runs your APP, the loading time will be 0.12 secs, for example.

What happened???
It’s great! At the second run, the NinevehGL automatically identified that those 20 3D files were already parsed before, so it will use the cached NGL binary files instead the original ones!

But this feature doesn’t stop here! Remember, NinevehGL was made to make everything be simple! So, let’s suppose your application loads the 3D files through the internet connection instead loading it locally. No problems! NinevehGL compares the modification date of the files and choose the most recent. Imagine this:

1. Your APP loads the 3D files from the internet, because you want to make updates without submeting a new APP to APP Store.
2. An User downloads your APP. First run, 20 3D files in 12 secs.
3. In the next times, NinevehGL will use NGL binary files, loading them in 0.12 secs.
4. You make an update and upload some new 3D files.
5. All the users that have your APP will receive the update, with 10 new 3D files replacing the old ones, so NinevehGL will parse the new downloaded files in 10 secs and it’ll generate new NGL binary files.
6. In the next runs of your application, NinevehGL will use the new generated NGL files and the loads will happen in 0.08 secs.

I cry of happiness every time I explain this routine. It’s amazing, wonderful, outstanding, great, it’s… it’s… “magical”! Well, this is just my opinion about it. But I’m sure you can imagine how great would be to the final user: “At the first run, a loading of few seconds, next times, no loading? Where is the loading of this APP?”. The loading happens in a snap.

NinevehGL automatically saves NGL binary files locally.

The last thing I want to say about this feature is the optimization. Again, the NinevehGL was made to be EXTREMELY FAST! So, if you load a NGL binary file directly, it will not create a cache file to it, because it is already optimized, it’ll just create cache from non optimized files (OBJ and COLLADA). Now you know how to convert your 3D models to NGL binary file, you can convert them on-line, at the official NinevehGL website, or can just run your application in the simulator and take the binary file generated at the first run and find the file at /Library/NinevehGL!

In order to avoid confusion, it’s important to say that NinevehGL will not be responsible to manage the internet connection for the file downloading. So you can’t inform an URL to load a 3D file. What the NinevehGL will make is manage the files locally.

Custom Shaders

The NinevehGL was made to work with OpenGL programmable pipeline. So obviously it’ll create its own shaders, based on the materials (loaded from 3D files or materials created directly in the code). Well, the most fun part of the programmable pipeline is exactly the shaders, so will NinevehGL block us of using our own shaders? Of course NOT! NinevehGL is absolutely user friendly, it’s flexible!

The other engines working with programmable pipeline are used to give to us only two choices. Working only with your very own shaders or working only with their shaders! Well, this is not a choice, but rather it’s a dilema! But, the NinevehGL offers another way, a third way: “all of above”! Yes, your read it right, NinevehGL has an API Shader capable to integrate two or more shaders into only one program. WOW, what that means?

That means you can create your own shaders, just as you are used to, implementing your own lights effects, materials or anything else you want. Then you pass your shaders to the NinevehGL and it’ll integrate your custom shaders with the shaders generated by the NinevehGL materials. No conflicts, everything works fine together!

This works for Vertex Shader and/or Fragment Shader. An important thing to say is that there are no special constraints, you can even use a shader that you already have, with many variables inside, many functions and the main function. The NinevehGL Shader API will interpretate the code inside the shaders and will fuse all together.

NinevehGL can merge your custom shader with its own shaders.

Calculating Normals and Tangent Space

This is the robust side of NinevehGL. It’s very fast even in the hard processing scopes, like calculations of normals and tangent space for imported models. Normals and tangent space are very important to generate real time lights, reflections, bump effects and many other things. NinevehGL automatically calculates the normals and generates the tangent space for each mesh that needs it.

NinevehGL automatically generates normals and tangent space, if needed.

NinevehGL can automatically generate some effects, like specular lights, ambient light, emissive light, bump map, reflections and other ones specified by NinevehGL materials. NinevehGL doesn’t impose any limitations to you, it just works around the limits of the OpenGL. So, in the iOS, for example, you can work with very high mesh models. Obviously this is not a common situation, but sometimes it could be better to see high mesh models on the screen. Even to these high meshes, NinevehGL will continue to produce the normals, tangent space, specular lights, bump maps, reflections and everything else you need.

NinevehGL also works with very high mesh models.

Now I want to show you some other features that are also great, but I’ll not talk in deep here.

Optimizing Textures

Just to keep it fresh on our minds, NinevehGL was made to operate in the maximum performance. So even the loaded textures will be optimized. If you choose to work with an opaque OpenGL Layer (the default), the textures could be optimized to the format RGB_5_6_5, which is the best choice without alpha channel. If you choose to work with transparent layer, the textures could be optimized to the format RGBA_4_4_4_4.

The optimization process will happens automatically and you don’t need to worry about it. Even to PVRTC compressed texture format, these optimization could happen, if needed to. The textures, as any other external file, can be loaded from any path locally and the NinevehGL is ready to manage the local paths to you.

Plus, just as the 3D files, textures has a kind of “cache”, that means if you load an image more than one time, the NinevehGL will manage the load locally and will not spend more memory with the same cached image. This is a very important optimization, mostly in cases when a 3D file use the same image for many things, like to the ambient, diffuse and specular maps.

Many other things

There are in NinevehGL many other important features, but I’ll just point it because other engines also have these features, so it’s not that exclusive. Like:

• Cameras with projections (perspective and orthographic).
• Customizations of OpenGL behaviors, like the render buffers.
• Simple API to make 3D transformations (like obj.rotateX += 1.0).
• Always use OpenGL optimization features, as Buffer Objects, array of structures, optimized texture formats, etc.

One great thing to talk a little bit more is about the auto-corrections. It’s very common that the 3D modelers create their models with different scales, I mean, a 3D file which contains a spoon could has the range of vertices going through -1000.0 to 1000.0 and other file which contains a house with the range of -0.5 to 0.7, for example. By placing both files in a 3D softwares you probably will see a house at the side of the empire state in format of a spoon.

For a 3D softwares this is not a big deal, because there are an infinity work space and you can simply reorganize your objects visually. But for a programming language, this is a problem and gets even bigger if the object is so big or so far that you can’t even see it on the screen.

The NinevehGL auto-correction can solve this problem for you, because it can normalize the vertices positions to fit on the screen (or in a specific range you want). By doing so you can control the size of your models without having to export it again from the 3D software. So in the case of the house and the spoon you could set the auto-correction to fit the house at the range -10.0 to 10.0 and the spoon to -0.1 and 0.1, simple as that (one line of code).

Outside the Scope

What you should not expect from NinevehGL? Here is a list:

• The NinevehGL is not a game engine, it’s a 3D engine.
• So you shouldn’t expect physics controllers or sound controllers. You should construct this kind of stuff by yourself.
• No animation, YET! Animations is very important in the 3D world, but the first version of NinevehGL will not come with this feature. (Animations = Character Rigging + Bones)
• No collisions, YET! There are two techniques to deal with collision, bounding box and bounding mesh. In this version, NinevehGL will not implement any of those.

Obviously any item in this list can change, because the NinevehGL is flexible. Consider the above list as an “out of scope” list only for the first version of the NinevehGL.

Conclusion

I’m sure you have now a solid opinion about NinevehGL, good or not, so let me know what you think, post a comment bellow, send me an email, tweet me… anything.

Again, I don’t want to make promisses, but I really want to see this engine released soon. So I’m sure it will come in the first half of this year (2011). I’m just finishing some details now and organizing everything for the first release.

Thanks for reading,

And see you very soon!

NinevehGL will come in the first half of 2011.