XNATools

OpenCP 1.0.0 is now released. It is available at:
OpenCP-src-1.0.0.zip

To the extent possible under law, Russell Klenk has waived all copyright and related or neighboring rights to OpenCP. This work is published from: United States.

As mentioned previously, OpenCP is structured around the concept of a content database. The content database is constructed during the content build process for a specific project and target platform combination. It maintains four different types of records:

Source content records are constructed by the build tool at the beginning of a build; source file records, target asset records and target file records are created by content conditioners during the build process. Links between the records are also constructed during the content build process. For example, a source content record maintains references to the source files used as input and the target assets produced as output. The content database allows an application to ask the following questions:

Maintaining the links between records can be a bit tricky, so OpenCP attempts to simplify the process. An application can query the content database for source content and target content by name or index, and can query for source and target file references by path or by index.

The nice thing about OpenCP is that it makes this information available to any application that wishes to use it. The OpenCP library includes a .NET implementation; a C/C++ implementation is coming soon.

The following diagram gives a basic overview of the OpenCP build process. The blocks and arrows in red are application-defined portions, while the blocks and arrows in blue are things implemented by the OpenCP library.

The process begins with application-defined source content, composed of source files and possibly (likely) some metadata. The files and metadata are in application-defined formats; OpenCP places no restrictions on the data or the file formats. This differs from previous iterations, where the content pipeline library defined the metadata format.

The build tool takes this data and transforms it into a series of one or more OpenCP source content definitions, which are persisted in the OpenCP content database. Each source content definition has a unique content ID (a simple string name) within the database, zero or more references to source content files, and some optional metadata, represented in a generic key-value form (both keys and values are strings.) Content processor implementations can access this metadata during the build, providing a mechanism for passing variable parameter data to control the content build process.

Each source content file reference maintains a hash of the file contents, last write timestamp, size in bytes, and path (relative to the application-defined project root – this allows projects to be easily rebased.) OpenCP uses this information to optimize the build process, so that only modified content gets rebuilt. If a piece of source content has not changed (and all of the associated target files still exists, and are unmodified) then the source content is not rebuilt, and any output is simply reconstructed from information stored in the content database.

The actual content build process is not controlled by OpenCP anymore. Instead, the application-defined build tool can load the content database for the build target platform, and query the database to determine whether a particular piece of source content needs to be rebuilt. Again, this allows the project to fully define the content build pipeline; OpenCP just takes care of the parts common to all content build pipelines.

Content processors in OpenCP serve largely the same purpose as they did in previous iterations; they receive a source content definition and produce zero or more target assets (each with zero or more target files.) Different content processor implementations can be specified for different target platforms, so it is now possible to specify a content processor that outputs textures using DXT compression for OpenGL and Direct3D-based platforms, but outputs textures in PNG format for the Flash platform (as an example.) The same content processor implementation can also be supplied for multiple target platforms.

Deferred content processors are a new feature. Deferred content processors are intended to perform any ‘global optimization’ portions of the build process. A great example is building texture atlases or sprite sheets, where information about multiple source content items is required to produce output textures with the least amount of wasted space. Execution of deferred content processors is an optional step in the build process.

The content database is the key to enabling all of this functionality, and the information it tracks will be the topic of the next post.

Tomorrow, I’ll be posting the first release of OpenCP, the next iteration of the xnatools content processing toolset. Work on the OpenCP library was completed in November of 2010 (why does everything seem to get done in November?). There are quite a few major differences between OpenCP and previous iterations, not the least of which is that it is cross-platform, running on the Microsoft .NET Framework on Windows, and Mono on Linux and OS X. It still targets the .NET 2.0 runtime. Since it has absolutely nothing to do with XNA, I figured it’s time to drop the XNATools association.

Unlike previous releases, OpenCP provides only the foundation for a content pipeline. I realized that for the library to be useful, people are going to need to provide their own content definition formats and customize the build system to their workflow. OpenCP provides the basic content build framework, including dependency and reference tracking, minimal rebuilds, pluggable content conditioners, target platforms, and so on.

The biggest change is the the switch to focusing on a content database model. Upcoming posts will provide more information on the content database and how it is used by OpenCP and build systems. The major benefit is that a content database can be loaded and dependency, reference, input and output information can be used by a number of tools, which when combined make up the full content pipeline. For example, the content packaging process can now be implemented entirely independently of the content build process.

Edit: OpenCP 1.0.0 is now released. It is available at:
OpenCP-src-1.0.0.zip

To the extent possible under law, Russell Klenk has waived all copyright and related or neighboring rights to OpenCP. This work is published from: United States.