2010 SPWebConfigModifications Applied

I have been noticing a few issues about SPWebConfigModifications and how they are applied.

Service Level SPWebConfigModification Applied

For instance if you have two web applications on your content service one on port 80 and other on port 8080. You can choose to have your web config modifications added to all of web applications by adding them at the SPWebService level.

SPWebService.ContentService.WebConfigModificaitons.Add(myWebConfigModification);
SPWebService.ContentService.Update();
SPWebService.ContentService.ApplyWebConfigModifications();

Advantages:

• Applied to all web applications at once
• Applied to new web applications when created
• Can be used in an auto activated farm SPFeatureDefinition

Disadvantages

• Will be applied to web applications set up by third party applications that may have special purposes which may damage these applications
• The actual SPWebModificaitons may reference specific solution based software that may not be deployed globally

Web Application Level SPWebConfigModificaitons

Alternatively it is possible to apply SPWebConfigModificaiton to a specific SPWebApplicaiton.

SPWebApplication myWebApplication;
:
myWebApplication.WebConfigModifications.Add(myWebConfigModification);
myWebApplication.Update();
SPWebService.ContentService.ApplyWebConfigModifications();

Advantages:

• Scopes to the WebApplication level SPFeatureDefinition
• Can be added to a WebApplication deployed SPSolution
• Can be added to a WebApplicaition feature that may be auto activated/deactivated

Disadvantages:

• Auto activate features always activate on create new web application.  You must make sure all necessary infrastructure pieces are in place to support the new web application as it bootstraps into IIS.

Summary:

In general the create new web application use case is often the most overlooked use case by developers.  SPWebService level SPWebConfigModificaitons will be applied to this use case.  Additionally WebApplication auto activate features will be applied to this use case.  As you program SPWebConfigModificaitons, make sure you do not block the farm administrator from creating new web applications.

Dangling SharePoint Features and Files at Upgrade

I recently upgraded from SharePoint 2010 RC1 to SharePoint 2010 RTM on my development machine.  Being a development machine there were several old projects that had been ripped from SharePoint without deactivating the features.  SharePoint tends to swallow these problems – although they show up in the logs.  Upgrade, on the other hand, is blocked by these problems.

I received several errors at upgrade time including errors like

[OWSTIMER] [SPContentDatabaseSequence] [ERROR] [5/6/2010 1:10:30 PM]: Found a missing feature Id = [11d1f820-10d9-48c9-bc44-4ae4439be635], Name = [FeatureName], Description = [], Install Location = [PackageName_FeatureName]

And

[OWSTIMER] [SPContentDatabaseSequence] [WARNING] [5/6/2010 1:10:31 PM]: File [Features\ PackageName_FeatureName \ModuleName\Files\Custom.css] is referenced [1] times in the database [WSS_Content], but is not installed on the current farm. Please install any feature/solution which contains this file.

[OWSTIMER] [SPContentDatabaseSequence] [WARNING] [5/6/2010 1:10:31 PM]: One or more setup files are referenced in the database [WSS_Content], but are not installed on the current farm. Please install any feature or solution which contains these files.

I was not surprised by the first error.  I had removed features without deactivating them, and how could psconfig know how to upgrade them if they no longer existed in the farm.  The second set of errors was unexpected and spoke to a new layer upgrade I had not expected.  The files that it was complaining about were documents that I have deployed to libraries using SharePoint modules.  The modules were gone, but the libraries existed.  How did it know that the document came from a SharePoint module and why does it care?

Further examination of the dbo.AllDocs table in SQL Server Management Studio turned up that it contains the columns “SetupPath” and “SetupPathVersion” which can tie a document in a library back to a feature deployment action.  Apparently, one of the activities psconfig does during upgrade is to check if the file in the library should be replaced by a newer version that had been deployed into the 14 hive.  I am not sure if that is done by comparing the matching SPPersistedFile within the config database or by comparing the document with the actual file on disk.   The SetupPathVersion is a tinyint so it is hard to understand what it maps against.

Regardless of the algorithm, it is pretty clear what upgrade was attempting to do.  This adds a new item to my list of upgrade functions that psconfig executes:

+ Check SPPersistedObject versions in the config database and call the IUpgradable and IUpgradable2 interfaces to get them to upgrade as needed.

+ Traverse the SPSite-SPWeb trees in the farm and upgrade each feature instance using the feature upgrade CAML along with any SPFeatureReceiver.FeatureUpgrading methods.

+ Examine all Library documents, determining if any come from features.  If they do come from features, check to see if a new version needs to be loaded from the 14 hive.

In my situation the upgrade failed and directed me to fix errors (shown in log) and try again. The error may say: “Please install any feature or solution which contains these files.” I thought this would be rather risky with my half upgraded farm. To complete my upgrade, I went to SQL Management Studio and deleted the problematic data from my content database.  I queried the dbo.Features table based on the feature IDs in the log and deleted those rows.  I queried the dbo.AllDocs table based on the SetupPath of files found in the log and deleted those rows.  I kicked off:

PS> psconfig -cmd upgrade -inplace b2b

Which completed successfully and I was upgraded.

 

SharePoint Developer Occasional Tips

This blog formally known as “SharePoint Developer Tip of the Day” is now “SharePoint Developer Occasional Tips”. The reality is that for a long time I had the idea that I would one day blog about many things SharePoint. I deal with it every day at work and I find out about many things that might be useful to others. The reality is that I am often too busy at work to get all those posts out. So I have changed the blog name to reflect reality. Part of the reason is I have been avoiding starting up again because there is the expectation that I must keep it going if I do. Now that expectation in gone, I can post whenever I feel the desire. The Google group which distributed the blog by email changed name also, but contains the original membership list.

Dave Graham

In-place Upgrade of Solutions in SharePoint 2010

The In-Place upgrade mechanism for SharePoint solutions now uses the feature upgrade mechanism to upgrade features already installed and activated in the farm. In this scenario an existing version of a solution exists within a farm with a set of active features. The goal is to upgrade it to a new version without retracting and redeploying. This will allow any active features to upgrade if needed.

Only one version of any solution will exist in the farm solution store. In the first step, a WSP that exists in the solution store is updated with a WSP file on disk. Then a job is run in the timer service and files in the WSP are deployed into the 14 hive on top of the existing files. (This is an update of the SPPersistedFile objects in the config database.) Finally psconfig must be called to upgrade features using the new 2010 feature upgrade system and to upgrade administrative SPPersistedObjects using the IUpgradable and IUpgradable2 interfaces.

The upgrade mechanism extends what was delivered in 2007. Either STSADM or PowerShell can be used to upgrade solutions stored in the farm solution store.

The STSADM approach:

CMD> stsadm –o upgradesolution –name <Solution name> -filename <path to new version of solution> -immediate -allowgacdeployment

CMD> stsadm -o execadmsvcjobs

CMD> psconfig -cmd upgrade -inplace b2b

The PowerShell approach:

PS> Update-SPSolution –Identity <Solution GUID ID or solution name> -LiteralPath <path to new version of solution> –GACDeployment true

PS> psconfig -cmd upgrade -inplace b2b

It is important to note that the “b2b” argument for psconfig. It indicates that the upgrade will action will be executed on a build to build comparison. With “b2b” any feature that has any positive version change will be upgraded. Alternatively the “v2v” argument can be used. The “v2v” argument will cause features to only be upgraded if the major and minor versions increase; it ignores the build and revision portion of the version string.

If the solution to be deployed contains assemblies for the GAC the –allowgacdeployment must be used with stsadm or –GACDeployment true with Update-SPSolution. Watch the time between the completion of stsadm or Update-SPSolution and the completion of psconfig. During this time the GAC will be running the new DLL assemblies while the features will still be configured with the previous version.

The upgrade is global to the farm and all installed components. The psconfig will search the farm for all items that need upgrade and upgrade them in place. This will include all pending SharePoint patches that have been installed as well as any third party solution upgrades that have been place in the system but to which the psconfig –cmd upgrade has not been run.

It is possible to end up with a partial upgrade. All activated features that can be upgraded will. Any that can’t be upgraded will be noted in an upgrade log. The psconfig tool will point you to the log in the event of failure.

I have performed many upgrades on solutions stored in the farm solution store during my testing.

The in-place upgrade for the new SPUserSolution sandbox solutions is very different. It happens automatically if you have set up the WSP file correctly with the same solution ID but a different hash code. Here is what Microsoft says:

An upgrade for a sandboxed solution is any solution that is uploaded to the solution gallery with the same solution ID of the version deployed to the site collection but with a different hash code. Incoming requests fail during the upgrade process. Any feature upgrade actions will be processed as well. During upgrade, the feature definitions for the existing solution are compared with the feature definitions for the new solution. Existing feature definitions are upgraded on the site and all subwebs are removed if they no longer exist in the new solution. All new feature definitions are activated on the site. However, upgrade will fail if an existing feature definition is a newer version than the feature definition that is included in the new solution.

I have yet to try it and see if it all works as they claim.

Features are Versioned and Upgradable on SharePoint 2010

One of the big sticking points with SharePoint 2007 was that Microsoft failed to adequately address was how to update between versions of third party SharePoint applications. Microsoft has addressed much of this introduction of feature versioning in 2010.

Starting with SharePoint 2010, the versions of all features activated are stored. Farm and web application features versions are stored in the config database. Site collection (SPSite) and site (SPWeb) features versions are stored in the content database. Features shipped without version numbers default to version 0.0.0.0.

With the in-place upgrade process provided in psconfig.exe, the farm is scanned and the versions of all features activated are compared with the versions of the feature definitions installed. Any active feature that is not up to the current feature definition (SPFeatureDefinition) version has the opportunity to upgrade. How that upgrade occurs depends on a new block of CAML XML in the feature definition under the new UpgradeActions element.

There are several types of UpgradeActions. They include CustomUpgradeAction, ApplyElementManifests, AddContentTypeField, and MapFile. The most flexible of these is CustomUpgradeAction for which you can assign a SPFeatureReceiver. When you do that it will invoke a new method FeatureUpgrading on the SPFeatureReceiver.

Some upgrade code paths must always run. An example of this would be having a base schema for a content type which never changes (because it was released in the past), but always AddContentTypeField regardless of whether deploymenting a new current version or an upgrading to the current version.

At other times, you must use the VersionRange element. This element specifies the affectivity of the upgrade code. An example would be a new module added to older versions.

Over the next few days I will post some examples of using feature upgrade.

PowerShell Setup for SharePoint 2010 Diagnostics

STSADM still ships with SharePoint 2010 but has been replaced with PowerShell as the preferred scripting engine for SharePoint. I rarely write SharePoint scripts, but I do often use these tools for testing. Lately I have been testing a lot of upgrade scenarios. To make the best use of PowerShell as a SharePoint diagnostic environment I have a few tips.

PowerShell Community Extensions – Get and install these extensions as they will make your PowerShell environment much nicer. Once installed, edit the (My)Documents\WindowsPowerShell\Profile.ps1 file to uncomment the Visual Studio environment variables. It is necessary to add ’10.0’ at the front of the comma separated list for “VisualStudioVersion”.

Auto Load the SharePoint Extension – Add the following line to your PowerShell profile.ps1 file to automatically load the SharePoint extension for PowerShell in every shell you launch.

& "C:\Program Files\Common Files\Microsoft Shared\Web Server xtensions\14\CONFIG\POWERSHELL\Registration\sharepoint.ps1"

PowerShell Threading Model and Extension Object Disposal – Each line entered in PowerShell is run by a single thread. This creates some problems because many SharePoint objects are attached to IIS COM objects and need disposal. The SharePoint extension methods for PowerShell deal with this to some extent. You get objects that appear to be the SharePoint .Net objects but are in some way disconnected from the IIS backend. The result is that some of the methods and properties of the object may not function. When you call these methods you may get a response that the object has been disposed.

Workaround for the Extension Object Disposal Problem - To get around this problem, use PowerShell to instantiate the .Net objects directly. This was possible with SharePoint 2007. When doing this, the object and the disposal should be contained in a single line of PowerShell (one thread). Here is an example of the threading model in practice:

PS> [Guid]$featureId="6d44695f-ca99-4c27-9b6b-2a39c62357a1"
PS> $site=new-object Microsoft.SharePoint.SPSite("http://server/sitecollection"); site.RootWeb.Features[$featureId];$site.Dispose()

This example can be used to show the version of a particular feature. I have used this a lot while studying the new feature upgrade capabilities in SharePoint 2010. Upgrade is a subject for another time.

Finding the Right Cmdlet – Microsoft has yet to publish the PowerShell documentation for the SharePoint extension. It is possible to dig the list of Cmdlets from looking at the SharePoint API documentation. I find it easier to just play with them in PowerShell. I use the following help function to make it easier to find the Cmdlets I need. You can put this in your PowerShell profile.ps1 file to make it available.

function findhelp([string]$part)
{
help > c:\temp\helpjunk.txt
$lookingFor = ".*" + $part + ".*"
cat c:\temp\helpjunk.txt where {$_ -match $lookingFor}
}

New VS Import WSP Project Speeds SharePoint 2010 Development

In SharePoint 2010, site templates are now saved as a WSP files. In addition, the new SharePoint tools for Visual Studio 2010 allow the contents of a WSP to be selectively imported. This allows the developer to quickly setup new SharePoint content using the SharePoint web UI or SharePoint Designer 2010. When the developer has gotten as far as possible with the SharePoint web UI or SharePoint Designer 2010, the structure can be imported into Visual Studio to add customization. The approach has saved me a great deal of time.

Here is how to do it:

1) Do as much design right in the SharePoint web UI and SharePoint Designer as possible until ready to transfer the work to Visual Studio.

2) Click on the Site Actions in the upper left hand of the sites root page. Select Site Settings from the drop down.

3) Under the heading Site Actions click on Save site as template

4) The Save Site as Template page should appear

a. Enter the File name for the template. Do not add the “.wsp” to the name as it will be added automatically later at download.

b. Add the Template name and Template description.

c. If custom pages have been created select the Include Content option. Custom pages created in the web UI or SharePoint Designer and placed them in Document libraries can be saved with the library by selecting this option. Note that selecting this option also includes a lot of other content data that may generate more cleanup work when importing into Visual Studio.

d. Click OK to save the template

5) Once the site template has been saved it will appear in the User Solution Gallery (at the site collection level). This is the location where the new SharePoint 2010 Sandbox Solutions are loaded and deployed. Upon completion, click on the link provided to the user solution gallery.

6) At the user solution gallery, do not activate the template. It is only passing through this location on its way to Visual Studio. Download the template simply by clicking its name

7) Start Visual Studio 2010. Click on New Project…

8) In the Installed Templates pain on the left select Visual C#->SharePoint->2010. Then select Import SharePoint Solution Package. Enter the name and location and then click OK.

9) At the prompt for the template file, browse and select the WSP file.

10) VS gives the opportunity to select what components within the WSP that are desired to be import into the project. This is where the real power of the tool lies. It is possible to import as little as a single field or list definition, or as much as the entire site template. If selectively importing only a portion of the WSP, it may take several tries to get the right combination of components to satisfy component dependencies. In general, if customizing on top of a common template such as Team Site parts like the common column definitions need not be imported, only the custom columns. Also, as far as I can tell, the property bags associated with the objects would rarely be needed.

11) After import there will be a project with a number of features and modules and other SharePoint components. The import makes no attempt to manage component dependency. Turning around to build and deploy the new project may not work. Check the packaging to ensure that components are deployed in the correct order to satisfy any dependencies. This can take some time if there are a lot of lookup fields defined in the project.

This approach lends itself to what I call the minimize costs methodology for SharePoint:

• Customize in the SharePoint Web UI to the maximum extent possible.
• Then, customize in SharePoint Designer to the maximum extent possible.
• Finally, turn to Visual Studio for anything that could not be handled in the previous two approaches.