This week, I stumbled across a very interesting problem in SharePoint that could’ve easily put my entire design into jeopardy. Lucky for me, I can post now that I escaped this beast handily after only a few hours of absolute terror. For the benefit of anyone else who walks down this path, I’ll try to recount my experiences.

I have three different SharePoint projects going on right now. Among other concerns, I’ve recently been focused on our deployment strategy. It’s a good time for this, because I spent a few weeks mocking up a couple sites and developing some custom features. Now, we need to move those sites into our UAT environment from the virtual server on my laptop.

There are a few tools for doing this, namely the VSeWSS, Solution Generator, and WspBuilder. Until recently, I mostly used the WspBuilder and post build actions to deploy some of my simpler projects in Visual Studio. This strategy works pretty well for situations where you need to test an assembly over and over again. It was also the only option (until recently) if you had assets that needed to be copied outside of the FEATURES folder. On top of that, the 1.0 version of VSeWSS had some serious issues, so there wasn’t much good reason to switch strategies.

To deploy my mockup sites, I wanted to just basically create site and list definitions and move them over to the new server. I decided to give VSeWSS 1.1 CTP a try and see if I could use it to do this. While the CTP has some issues and challenges, I feel comfortable now saying that once you have worked with it for a while and understand how it behaves, it is overall a pretty good tool. The solution generator is ideal for replicating a site on the site server or packing up a site created entirely by a user. But, if you have a desire to leverage functionality like content types or list receivers, you’re going to end up to pushing its limits – often.

Today, I’m going to describe one of those limits, how it scared the living crap out of me for about two hours, and how you can rise to this challenge and overcome it.

If you are like most developers, you probably want to get traction in SharePoint as quickly as possible. This helps engage the customer and get the feedback process started so our designs are better. And, if you know SharePoint, you know it’s probably a pretty darn good idea to create Site Columns and Content Types instead of just slapping fields into a list; it’s the same amount of work, more reusable, and even required for certain functionality (like search) to work. It was with this idea in mind that I had created several lists with content types.

Of course, I want to re-use everything I created in the GUI, so I was now in the position of needing to create content type definitions based on what I’d already done during our discovery period. The Solution Generator that ships with VSeWSS does half this job for you, because content types are embedded with the list definition. They aren’t perfect of course, but I needed to clean up some field names anyhow. (IMHO, if you’re writing the definition file anyway, there’s just no good reason to have “_0x0020_” peppered throughout all your field names.)

So, I would use VSeWSS to create new (empty) content types, then paste the XML from the list definition, and massage it a bit to get the Fields and FieldRef elements in the right format. This isn’t a bad approach especially if you’re still learning your way around the wss.xsd schema, and it was a good opportunity to refine my approach and refresh my memory. (It had been over a year since my last official SharePoint gig, after all.)

In fact, this technique works fine, but there are a couple of things that aren’t entirely obvious at first glance that can trip you up when it comes to content types and feature definitions.

Firstly, if you’re content types are using the same GUIDs as the ones you created in SharePoint, then you are probably going to notice that your changes don’t always appear on the web site after you deploy. There are two reasons for this that I’ll explain later. The other thing you’ll run up against is that it doesn’t seem possible to use the ContentType element to define a type that inherits from another content type. There doesn’t appear to be any [documented] way to do it.

These two problems are actually related. To understand what is going on here requires a little background on how SharePoint manages Content Types.

SharePoint keeps track of Content Types in many places. They can have Site or Web scope, meaning that they are available in either the root site (top level web) or an individual sub web and its sub webs. Within each of these scopes, it is possible to derive a new type based on a parent type, even when that parent type is defined in a parent web site.

Actually, that’s a pretty nifty thing if you think about it, and as developers we want to be able to take advantage of that inheritance as often as possible to save us some work in both development and maintenance. Up until now, developers have tried to do this in their projects, but found that the only apparent ways to do so were either in the web UI or via the object model. This introduces another issue, in that once a content type is modified in the UI or API, it only gets its information from the content database SQL Server, and never from an XML element manifest. This would seem to rule out using features as a way to deploy new content types, unless of course you are writing a ton of code to create the content types programmatically, and we all know how much fun that can be!

Let me take a step back for a minute. Once you have content types defined in your site or web, you may have noticed that if you use them in a list, a mirror image of your content type will appear in each list. Actually, that’s kind of a cool feature too, because it may be that in one particular case you need to change the name of a field or some other property. Having the list based content type allows us to do that without affecting the parent and all the other lists that use our content types.

So, we have content types stored in the root site (both independent and inherited from other CTs), in the sub-webs (again, either independent or inherited), and in the lists themselves (always inherited). Let that last one linger in your mind for a bit, because that’s how I was able to figure this whole mess out.

I had noticed a while ago that when I use the Solution Generator to create a list definition, the content types always had IDs that were significantly longer that the ones generated if I created a new Content Type using VSeWSS 1.1 CTP. From the beginning, it had become pretty obvious to me that the latter of these was probably a GUID with some salt (“0x0100”) sprinkled in front of it. But, the GUIDs from the list definition were much longer – which was weird. I wondered a bit about this, and decided to use the GUIDs created by Visual Studio instead of the ones in the list definition, but I didn’t dig into it any deeper until today.

I’ve seen some folks have called these “magic GUIDs”, and some hacks have been documented where you can create a content type in the GUI and then steal these magic IDs to use in your Content Type manifest files, and the inheritance will follow them. Of course, any technology – sufficiently advanced or undocumented – will seem like magic. Obviously there isn’t some spirit that is following these funny ID attributes around and thus magically bestowing a parent-child relationship upon them. Or, is there? Could there be a Ghost in the Machine – literally?

Or, should that be "Re-ghost in the Machine"? It seems like all this talk of physical vs virtual content type definitions seems somehow eerily familliar. :-)

Anyway, I had an intuition that these mystery strings had something to do with the missing ability to define parent-child relationships, so I decided to do some digging and a few controlled experiments to see if I could unravel this inscrutable conundrum. Of course I’ve already spoiled this great detective story, because I already told you that I saved the day, disaster was averted, and I avoided being in a terrible mood for the rest of the week.

Firstly, these strange strings are made entirely of hex digits. This fact is made obvious by the leading “0x” at the start of the ID. Also, they are almost universally (but not always) some length that is close to a multiple of 32, which happens to be the length of a GUID with the dashes taken out. But, some are not. Well, if you keep in mind that 0x0100 and 0x000100 mean exactly the same thing, then things start to line up a little better. Allow me to demonstrate using the IDs of several ContentType elements that I generated in VS.net. I have inserted underscores to pad them out (not a legal substitution of course, but good for visualization purposes.)

So from this:

<ContentType Name="Item" ID="0x01006964b2a9ff5c4a299a55daf0ca370a79" Group="Development" Version="0"><FieldRefs /> </ContentType>

We get:

Item:          0x____0100 6964b2a9ff5c4a299a55daf0ca370a79

And so on…

Document:       0x__010100 28ec491792cd40469e5023279d495a74
Event:          0x__010200 0d11c8f44fee4d299fa7412976ced2f3
Issue:          0x__010300 d0ff4d76b975418f92f99b6380525b51
Announcement:   0x__010400 ca7fe46207ee4f678658688328478ac9
Link:           0x__010500 94746e4cec53476faf5c25b625375005
Contact:        0x__010600 d74d210576bb41d1ad7c2129f6b2f146
Task:           0x__010800 39f9376dd88d446fa540ae6b0aab80b2
Folder:         0x__012000 667c88b5ec704512acf75070fd6a9cb8
Discussion:     0x01200200 08993122a65146068f4acfc1f7be60f5
XMLDocument:    0x01010100 afce5e1e9dd84a7abc719987a7bd1627
Picture:        0x01010200 bc7bc4a9bb6c4dbe924eccebcf2e260a
WikiDocument:   0x01010800 8898737798e045a6bcbf8ae3617da0ac

So now, at least for the basic types, it seems pretty obvious that the first eight digits contain the base type that the content types is derives from.  This makes me ask myself all sorts of strange questions, like “What happened to 0x0107?” and “What other types that I don’t see here (Project Task List anyone?) could I exploit if I knew their codes?” But, my mission is clear, so moving on.

I also exported some Content Types from my own list definitions, and I observed some meaningful things about them. I will spare you having to stare at another list of random GUIDs, and simply explain what I found.

·         Once you line up the strings so that the first GUID always starts in the same positions, as above, the remainder of the string always follows a consistent format. Additional characters on the string consist of a “00” pad (0-FF being a single byte), followed by 32 hex characters (a GUID). This pattern will repeat for the remainder of the string.

·         I have seen strings with two or three GUIDs stored in this way. I assume that since the ContentTypeID field (in the content database in SQL) is a 512 byte binary block, that the developers anticipated needing to store up to 17 GUIDs in this way.

·         Assuming that your site is working, the “last” (low order) GUID of any ID attributes generated by exporting from it will be unique. No other content type will (or should) ever have that GUID.

·         The high order GUIDs will not be unique. Somewhere in SharePoint there will be another content type that has that GUID as a segment of its ID.

·         If you want to create an inherited content type as part of your feature, create its ID in the format “0x” + Base ID hex + Parent GUID + “00” Unique GUID. I have tried this and it works. If believe that if you want to create multiple inheritance, you do it like “0x” + Base ID hex + Grandparent GUID + “00” + Parent GUID + “00” Unique GUID, but I haven’t confirmed this. Also, don’t forget to take the dashes out.

·         List content types are simply inherited types that have the web/site content type as a parent. However, they do not appear to be stored in the ContentTypes table in SQL. Maybe they are embedded within the list itself?

So why would the SharePoint development team implement inheritance in this way and not as a property of the ContentType element (BaseType for example)? I don’t know the answer, but I suspect that it had something to do with a freeze on code near the end of development, or perhaps there were different teams relying on the schema and it couldn’t be easily changed. That much I can forgive, but this methodology was so unintuitive that it makes me laugh just thinking about it.

Here’s what Microsoft has to say about the ID attribute:

While I can’t find anything factually wrong about that statement, I still have to say that it seems like something important is missing. ^_^

I found this article Content Type IDs after the fact, and indeed is it a very helpful elaboration on what I've explained above. Also, it seems to indicate that in addition to mulriple "00"+Guid combinations at the end of the first GUID, you can append any hex string. "You can simply append two hexadecimal digits to the myDocument content type ID." In fact they say this is a good way to stay below the 512 character limit. I wonder however, if it actually works and how inheritance is implemented in this case. (Since they don't do this themselves when you create content types OoTB with the SharePoint GUI, I have to question the accuracy on this point.)

So how are the two issues that started this conversation related? Well, it turns out that what the Solution Generator is extracting along with lists are the List Content Types. It looks like I will have to build a tool to export the actual Content Type from the web itself, after which the List Definitions should work without any issues, but all the related gripes about content type disconnection will still apply.