XNATools

I’ve finished adding the final features to the main projects, but there’s still a lot to be done to make a real release – finishing up the base set of content conditioners (texture processing based on NVIDIA Texture Tools, geometry from COLLADA files), as well as installers for everything, some decent testing, and documentation.

To avoid a bunch of postings like this one, I’m targeting early September for the release, accompanied by a reboot of the site. Check back September 12th.

Russ

I was supposed to post an incremental update this weekend, but after some review and a lot of late-night work, I’ve decided that version 1.0 is feature-complete. So, instead of the incremental release, I’m going to spend the next week or so performing some testing and review of features, and release 1.0 beta 1 of the content tools software. The release will include source code as well as binary installers for Windows x86 and x64. Check back next weekend for the release.

I forgot to mention this in the previous post, but the use of mixed-mode assemblies like SharpSVN has required everything to be build for specific CPU’s (x86 and x64) instead of the generic AnyCPU. This required creating x86 and x64 configurations for each existing project. Additionally, since SharpSVN supplies both 32- and 64-bit binaries, the project file for the Subversion URI resolver had to be edited by hand. It wasn’t difficult, but it would have been nice if different assemblies could be referenced for different platforms in the Visual Studio UI.

If you build the source, you need to make sure that everything is being built for the same platform. Post-build events have been added to all projects to copy their output into a ‘.dist’ directory (in the same directory as the ContentTools.sln file) where they are organized by platform.

When the code stabilizes a bit more, and has been proven out by implementing various URI resolvers and content conditioners, I’ll start releasing x86 and x64 binary packages (MSI installers) for setting up client machines.

The next source update fixes some (significant) issues with loading of assemblies for URI resolvers, package builders and content conditioners. Additionally, it will include an example package builder, the Subversion URI resolver, and Content Studio will be updated with integrated build support.

Most of the work is already completed. I discovered the issues with assembly loading while working on the Subversion integration. There were a few major issues. First, there were a number of places where I was using Assembly.LoadFile() instead of Assembly.LoadFrom(). You can read about the differences on MSDN (here for example), but the important difference is that using Assembly.LoadFile() doesn’t search for dependencies in the same path as the assembly being loaded, whereas Assembly.LoadFrom() properly handles dependency loads.

I decided to use SharpSVN to help with the Subversion integration. This exposed the second major issue with the content pipeline’s handling of assemblies. Content Studio copies any referenced assemblies into a directory (Binaries) local to the solution. I was using reflection to load the assembly, and copy any assemblies referenced by the assembly into the Binaries directory as well. The problem is that a significant portion of SharpSVN is implemented as a mixed-mode assembly (mixed CLR and native code), and part as (I believe) a native-code exe. The reflection mechanism doesn’t report these references, and as a result, the Subversion URI resolver would fail to load because referenced DLL’s weren’t copied into the local Binaries directory. This would also be an issue with native code DLL’s that were P/Invoked, or native code command-line tools. All of these things are common occurrences during content processing, since you’ll often have an existing toolset you want to use, or it may just be easier to implement a portion of your pipeline in C/C++.

My solution was to allow the user to specify a set of additional files they want to be copied into the local binaries path when selecting the main assembly file. It’s not ideal, but it’s the only thing that’s (reasonably) guaranteed to work.

The third major issue has to do with different versions of referenced assemblies. For example, say you have assembly A and assembly B, both reference ContentPipeline.dll, but assembly A was built against version 1.0.0.0 of the DLL, while assembly B was built against version 2.1.0.0 of the DLL. Assembly B may not be compatible with version 1.0 of the DLL, but using the previous directory structure, only one version of the DLL could exist, likely breaking either assembly A or assembly B. The solution to this is pretty simple – each logical assembly group now gets its own subdirectory inside of the Binaries directory.

The intention behind copying all assemblies, executables, etc. to the local solution directory is to (a) allow the solution to be moved from machine to machine and (b) allow the project to be rebuilt with relative ease on different machines without having to install a bunch of different programs and utilities – everything is in one place, and the solution is self-contained.

The final major change is that the command-line content compiler has been split into a DLL and EXE. The DLL contains the meat of the compiler, while the EXE basically just parses the command line, fills out a parameter structure and passes it onto the DLL, which implements the actual build process. This allows the compiler to be embedded into applications (Content Studio) without having to spawn an external process.

As a side note, the next release also has the search functionality implemented in the Asset Definition File editor.

The update should be posted by the end of the week.

The next two major components in the content processing system are targets and packages.

Each content solution must define at least one target and at least one package in order to build anything. A content package is pretty simple – it’s just a collection of references to asset definition files (discussed later). Content target files are a bit more interesting.

Each content target definition specifies the information necessary to process and package assets for a specific target platform. A target platform would be something like “Windows PC DirectX-9″ or “Xbox 360″. It defines two critical pieces of information:

The next post will talk about the asset definition files that specify the set of assets and the parameters associated with them that are used during processing.

The content compiler code has been updated (minor updates have also been made to the ConditioningEngine class in the ContentPipeline project.) The content compiler should now print out full log information. Text is stored in an application resource file, and can be localized if necessary. The new code is available here.

The content solution is the top-level container in the content processing system. It defines four major items:

The raw XML of a content solution file looks like this:

<?xml version="1.0"?>
<ContentSolution
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
SolutionName="Test"
OutputPathName="Output"
UriResolver="XNATools.DefaultUriResolvers.BasicUriResolver">
<LocalSchemas>
<SchemaLocation>.\Schemas\AudioTypes.xsd</SchemaLocation>
</LocalSchemas>
<Targets>
<TargetDefinition>.\Targets\WINx86_DX9.cctgt</TargetDefinition>
</Targets>
<Packages>
<PackageDefinition>.\Packages\Level01.ccpkg</PackageDefinition>
</Packages>
<UriResolverAssemblyLocation>
.\Binaries\XNAToolsDefaultUriResolvers.dll
</UriResolverAssemblyLocation>
</ContentSolution>


When you create a new content solution in Content Studio, it creates the following directory structure for you (values in parentheses have user-configurable names):

Next time, we’ll take a look at content targets and content packages.

The latest release is available here.

The ZIP file contains five projects (Visual Studio 2008 SP1):

In the near future, I’ll post some updates for the content compiler and GUI tool. After those tools are updated, I plan on posting a DLL that provides subversion integration, along with an example packager DLL. The next posts will focus on the different components of the system (solutions, targets, packages and asset definition files) and how everything fits together.

So, I’ve been revisiting my content processing stuff yet again so I can use it for some Xbox and iPhone development. The good news is that I’m not re-writing the content processing engine itself – I think it’s a pretty good architecture that can meet the needs of anything I would need to do with it. And therein lies the problem. I was so focused on making the content processing code super-flexible that I never really took into account how easy it would be to use, particularly for a small team.

In a nutshell, you define the asset metadata format using XML schema files (because, you know, XML solves all of the world’s problems…not!) This way I didn’t have to make any code modifications to add new asset types. That’s all well and good, until you have to actually produce the XML that describes the assets. That’s not so easy to do by hand, but I had planned on building a GUI tool to handle all of that. Great in theory, but like many times before, the GUI tool ended up being more boring work than I was willing to do in my spare time. Lesson learned – technical flexibility is great, but it’s worthless if you can’t actually use it!

So, I’m back to work on a set of very minimalistic GUI tools that take away most of the pain for programmers and artists alike. Each utility takes a day or less to code, and usually only does a single thing well. It’s not ideal, but it’s doable, and that’s what counts. The next major post (not going to give a timeframe, since well, my posting track record sucks) will have an updated set of code with these tools integrated, as well as some other stuff, like a data packager and some sample conditioners. Right now, the set of tools are:

Obviously, it would be nice if all of these utilities (except for the compiler) were in the same app, but I just don’t have the time.

As promised, the code for my content processing solution can be downloaded here. This code depends on the XNATools Framework assembly, available here. This code only provides a framework for a generic content processing utility; it doesn’t specify things like project file formats, nor does it define any content processors. Sometime next week I’ll post some example projects that provide those features.

I’ll write up a description of the classes involved, the roles they play, and what needs to be done to apply this code to a real project tomorrow.