Dangling SharePoint Features and Files at Upgrade

I recently upgraded from SharePoint 2010 RC1 to SharePoint 2010 RTM on my development machine.  Being a development machine there were several old projects that had been ripped from SharePoint without deactivating the features.  SharePoint tends to swallow these problems – although they show up in the logs.  Upgrade, on the other hand, is blocked by these problems.

I received several errors at upgrade time including errors like

[OWSTIMER] [SPContentDatabaseSequence] [ERROR] [5/6/2010 1:10:30 PM]: Found a missing feature Id = [11d1f820-10d9-48c9-bc44-4ae4439be635], Name = [FeatureName], Description = [], Install Location = [PackageName_FeatureName]

And

[OWSTIMER] [SPContentDatabaseSequence] [WARNING] [5/6/2010 1:10:31 PM]: File [Features\ PackageName_FeatureName \ModuleName\Files\Custom.css] is referenced [1] times in the database [WSS_Content], but is not installed on the current farm. Please install any feature/solution which contains this file.

[OWSTIMER] [SPContentDatabaseSequence] [WARNING] [5/6/2010 1:10:31 PM]: One or more setup files are referenced in the database [WSS_Content], but are not installed on the current farm. Please install any feature or solution which contains these files.

I was not surprised by the first error.  I had removed features without deactivating them, and how could psconfig know how to upgrade them if they no longer existed in the farm.  The second set of errors was unexpected and spoke to a new layer upgrade I had not expected.  The files that it was complaining about were documents that I have deployed to libraries using SharePoint modules.  The modules were gone, but the libraries existed.  How did it know that the document came from a SharePoint module and why does it care?

Further examination of the dbo.AllDocs table in SQL Server Management Studio turned up that it contains the columns “SetupPath” and “SetupPathVersion” which can tie a document in a library back to a feature deployment action.  Apparently, one of the activities psconfig does during upgrade is to check if the file in the library should be replaced by a newer version that had been deployed into the 14 hive.  I am not sure if that is done by comparing the matching SPPersistedFile within the config database or by comparing the document with the actual file on disk.   The SetupPathVersion is a tinyint so it is hard to understand what it maps against.

Regardless of the algorithm, it is pretty clear what upgrade was attempting to do.  This adds a new item to my list of upgrade functions that psconfig executes:

+ Check SPPersistedObject versions in the config database and call the IUpgradable and IUpgradable2 interfaces to get them to upgrade as needed.

+ Traverse the SPSite-SPWeb trees in the farm and upgrade each feature instance using the feature upgrade CAML along with any SPFeatureReceiver.FeatureUpgrading methods.

+ Examine all Library documents, determining if any come from features.  If they do come from features, check to see if a new version needs to be loaded from the 14 hive.

In my situation the upgrade failed and directed me to fix errors (shown in log) and try again. The error may say: “Please install any feature or solution which contains these files.” I thought this would be rather risky with my half upgraded farm. To complete my upgrade, I went to SQL Management Studio and deleted the problematic data from my content database.  I queried the dbo.Features table based on the feature IDs in the log and deleted those rows.  I queried the dbo.AllDocs table based on the SetupPath of files found in the log and deleted those rows.  I kicked off:

PS> psconfig -cmd upgrade -inplace b2b

Which completed successfully and I was upgraded.

 

In-place Upgrade of Solutions in SharePoint 2010

The In-Place upgrade mechanism for SharePoint solutions now uses the feature upgrade mechanism to upgrade features already installed and activated in the farm. In this scenario an existing version of a solution exists within a farm with a set of active features. The goal is to upgrade it to a new version without retracting and redeploying. This will allow any active features to upgrade if needed.

Only one version of any solution will exist in the farm solution store. In the first step, a WSP that exists in the solution store is updated with a WSP file on disk. Then a job is run in the timer service and files in the WSP are deployed into the 14 hive on top of the existing files. (This is an update of the SPPersistedFile objects in the config database.) Finally psconfig must be called to upgrade features using the new 2010 feature upgrade system and to upgrade administrative SPPersistedObjects using the IUpgradable and IUpgradable2 interfaces.

The upgrade mechanism extends what was delivered in 2007. Either STSADM or PowerShell can be used to upgrade solutions stored in the farm solution store.

The STSADM approach:

CMD> stsadm –o upgradesolution –name <Solution name> -filename <path to new version of solution> -immediate -allowgacdeployment

CMD> stsadm -o execadmsvcjobs

CMD> psconfig -cmd upgrade -inplace b2b

The PowerShell approach:

PS> Update-SPSolution –Identity <Solution GUID ID or solution name> -LiteralPath <path to new version of solution> –GACDeployment true

PS> psconfig -cmd upgrade -inplace b2b

It is important to note that the “b2b” argument for psconfig. It indicates that the upgrade will action will be executed on a build to build comparison. With “b2b” any feature that has any positive version change will be upgraded. Alternatively the “v2v” argument can be used. The “v2v” argument will cause features to only be upgraded if the major and minor versions increase; it ignores the build and revision portion of the version string.

If the solution to be deployed contains assemblies for the GAC the –allowgacdeployment must be used with stsadm or –GACDeployment true with Update-SPSolution. Watch the time between the completion of stsadm or Update-SPSolution and the completion of psconfig. During this time the GAC will be running the new DLL assemblies while the features will still be configured with the previous version.

The upgrade is global to the farm and all installed components. The psconfig will search the farm for all items that need upgrade and upgrade them in place. This will include all pending SharePoint patches that have been installed as well as any third party solution upgrades that have been place in the system but to which the psconfig –cmd upgrade has not been run.

It is possible to end up with a partial upgrade. All activated features that can be upgraded will. Any that can’t be upgraded will be noted in an upgrade log. The psconfig tool will point you to the log in the event of failure.

I have performed many upgrades on solutions stored in the farm solution store during my testing.

The in-place upgrade for the new SPUserSolution sandbox solutions is very different. It happens automatically if you have set up the WSP file correctly with the same solution ID but a different hash code. Here is what Microsoft says:

An upgrade for a sandboxed solution is any solution that is uploaded to the solution gallery with the same solution ID of the version deployed to the site collection but with a different hash code. Incoming requests fail during the upgrade process. Any feature upgrade actions will be processed as well. During upgrade, the feature definitions for the existing solution are compared with the feature definitions for the new solution. Existing feature definitions are upgraded on the site and all subwebs are removed if they no longer exist in the new solution. All new feature definitions are activated on the site. However, upgrade will fail if an existing feature definition is a newer version than the feature definition that is included in the new solution.

I have yet to try it and see if it all works as they claim.