---

You may also be interested in: Documentation Toolkit for SharePoint by Acceleratio Ltd.

---

 

Editor’s note: Contributor Stephane Eyskens is the founder of Silver-IT. Follow him @stephaneeyskens

As you might have noticed, AppFabric is now one of the pre-requisites when installing SharePoint 2013. SharePoint uses it for workflows and activity feeds.

When I noticed that AppFabric was part of the pre-requisites, I immediately thought of trying to figure out whether SharePoint would give developers new APIs to leverage AppFabric’s distributed cache capabilities. As SharePoint developers, in the 2007/2010 versions, when it came to caching we always had to either use ASP.NET cache (mostly inproc) or rely on third parties such as NCache or AppFabric but we had to install/configure those ourselves as they were not integrated in SharePoint.

Unfortunately, at the time of writing, Microsoft did not release any public API enabling us to benefit from a Farm-Wide caching system. Indeed, the classes leveraging AppFabric are all marked internal, so there is no public API. However, the good news is that we know AppFabric will be deployed on every SharePoint setup, therefore, we can start using it in the context of SharePoint using the AppFabric client DLLS.

I built a wrapper that you can download here http://sptoolbasket2013.codeplex.com/. Its purpose is to ease the use of the AppFabric layer in the context of SharePoint 2013. I should add that this is currently in an experimental stage since I’ve never used it for a real project and I don’t even know whether this wrapper would be supported by Microsoft :).

I’ll describe all the testings I’ve made with this wrapper & AppFabric in SharePoint 2013.

Table of contents:

Topology used for my testings
Services
Checking the AppFabric configuration
The AppFabric Way
The SharePoint way
Description of the wrapper
Getting started with the wrapper
Cache Invalidation
Local Cache
Different running contexts
Distributed Cache Accross the SharePoint Farm
Cache Distribution
Testing HA
One step further
Locking
Serialization
Working with regions
BulkGet
Cache Search
Cache Exceptions
Real World Scenario
Conclusions

Topology used for my testings

In order to test a distributed cache system, you really need to be in a farm configuration, stand-alone install is not sufficient. I built an environment on CloudShare which was made up of the following:

- A domain controller hosting the SharePoint services, WFE and SQL which I’ll refer further to as C6208759806
- A WFE that I joined to the existing farm which I’ll refer further to as C7763875784

This way, we’ll really see if the cache is indeed distributed :).

Services

In order to use distributed caching, you need to ensure that the Distributed Cache Service is started on both servers as shown below:

A corresponding windows service should also be up & running :

All the above is normally done automatically upon SharePoint installation but I prefer to mention it in case something went wrong during setup.

Checking the AppFabric configuration

The AppFabric Way

You can easily check if the AppFabric cluster is correctly configured by opening the PowerShell window labelled Caching Administration Windows. Then, you can type the following commands:

which will give you information about the AppFabric cluster configuration. One more command will give you information about SharePoint’s usage of AppFabric:

As you can see from the above screenshots, AppFabric listens to specific ports other than 80/443. Firewalls must be configured accordingly

The SharePoint way

Open the SharePoint Management Shell, you’ll find the AppFabric commands + some specific SharePoint cmdlets. They are listed in the below table:

and the one you should always start with : Use-CacheCluster.

Description of the wrapper

As stated in the introduction, the wrapper is aimed at facilitating the usage of AppFabric within SharePoint. It’s not making a complete abstraction of the underlying AppFabric layer but helps in creating factories according to the SharePoint config & topology.

It does the following:

• Allows you to work also with SharePoint 2010 providing you have installed AppFabric. The wrapper comes up with a configuration page in order to allow you to specify which servers are in use
• Detects which SharePoint servers are running the distributed cache service and adds those when creating the factory
• Makes use of SharePoint’s default cache area that is created automatically upon SharePoint installation
• Allows you to easily enable local cache
• Allows you to easily enable local cache notifications providing the cluster is correctly configured

Note that you can specify explicitely to use another cache than SharePoint’s default one. However, that cache must be created by an administrator using the PowerShell cmdlet New-Cache prior to consuming it from code.

Getting started with the wrapper

Using the wrapper, the simpliest way of adding an item to the cache is this one:

---

---

You first need to add a reference to the following DLLs:

• SilverIT.SharePoint.AppFabric
• Microsoft.ApplicationServer.Caching.Client
• Microsoft.ApplicationServer.Caching.Core

The wrapper can be used with the default settings that are calculated dynamically whenever a new factory gets instanciated. However, this operation is quite heavy. The wrapper offers two configuration modes :

• Dynamic : you can make use of the SPCacheConfig class to specify the behavior (localcache, named cache, timeouts etc..)

• Static: you can use a specific configuration page where you define once for all the parameters that the wrapper is supposed to use. This page looks like this:

  and what you specify there is persisted in a SPPersistedObject

Note that the latter approach takes priority over the dynamic one. For a full explanation of the wrapper and its configuration, go visit the documentation on the CodePlex project

Cache Invalidation

All the objects stored in the cache will expire after the time specified in parameter to the Add() method which has a few overloads and one of them allows you to specify the time to live for a specific object. So, if you add an object the following way:

---

---

It will remain in cache for 5 minutes. If you add an object to the cache the following way:

---

---

It will use the default TTL of the SharePoint cache.

Of course, if the objects get modified, they will also get invalidated.

Local Cache

AppFabric provides a way to enable local cache. When local cache is enabled, if a consuming process requests a specific object, it keeps it in its own memory for further uses. The advantage is that subsequent accesses to the cached objects are faster since you avoid an extra roundtrip to the cluster cache and you avoid the serialization process that occurs along with the roundtrip. To observe that behaviour, you can easily write a small console application, you enable local cache programmatically (see below) and you can for instance add 10 objects of +- 1MB each to the cache, after that, you read them 1 by 1 in a loop with a sleep of 2 seconds from another instance of that console app and you monitor the memory usage with the Task Manager. You will clearly see that every 2 seconds, the memory usage of this console app will increase of more or less 1MB.

If however, you do not enable local cache and do the same "exercise", you’ll notice that your memory will increase/decrease/increase/decrease etc…and not constantly increase anymore.

In SharePoint, it’s the worker process w3wp.exe that will store locally cached objects. To enable local cache using the wrapper, you just have to do the following:

---

---

this will by default use Timeout based expiration policy. I actually created 5 overloads which are :

Timeout based expiration with default object count and default timeout

Timeout based expiration with specific timeout

Timeout based expiration with specific timeout & specific object count

Notification based poclicy with specific timeout & specific polling time

Notification based poclicy with specific timeout, specific polling time and specific object count

If you want to use notifcation based expiration, you can do this:

---

---

With the above example, the system will check every 5 minutes whether there is a notification for locally cached objects. If nothing has changed, it will keep using the locally cached object until the timeout for the locally cached object is reached.

Note that you often see on the web timeout based or notification based. In reality, it’s actually timeout based and notification based because whatever method you’re using, you’ll have to specify a local cache timeout even if you’re using the notification based policy.

In the context of SharePoint, if you want to use the notification based expiration policy, you can enable it the following way with PowerShell:

---

---

Please note that this might not be supported by Microsoft and make sure you carefully test this configuration on dev/integration/acceptance environments instead of rushing in production :). Note that when using Notification based policy, the timeout specified in the local cache config is still used by the system meaning that some items could be evicted from the local cache although no notification was triggered.

Different running contexts

As you might have noticed from the above examples, AppFabric can be used from non-web contexts.

This offers the following big advantages, in the context of SharePoint :

• You can build time consuming cached object from within a Timer Job for instance
• Your cached objects can be shared accross different SharePoint web apps
• Your cached objects will reset to an application pool crash, IISRESET
• The wrapper facilitates the usage of AppFabric
• The SharePoint Infrastructure offers a good abstraction layer to manage AppFabric and facilitates admin’s life
• Providing you’re running in a farm context and that you started the DistributedCache service on more than 1 server, you’ve got a HA cache system (read more on that below)

Distributed Cache Accross the SharePoint Farm

To perform the below tests, I wrote a very basic SharePoint WebPart that I added on a page served by my servers C7763875784 and C6208759806 having both the WFE role enabled.

Consider the following code (reusing the cache wrapper explained earlier) :

---

---

in which I store a simple string in the cache, I first check whether it’s already in the cache, if not I add it.

Cache Distribution

I went to the server C6208759806, browsed the home page and added the webpart which gave me the following result:

As expected, the string was not in the cache so the webpart has added it. After a page refresh, you get the following:

again, here, expected behaviour since now the item could be obtained successfully from the cache.

Then I went to server C7763875784 and just browsed the home page which returned this:

As you can see, the request has been served by C7763875784 not by C6208759806 where we inserted the item in the cache and the webpart could read the value without any problem. This simple basic test shows that the cache looks to be indeed correctly distributed. With the OOB ASP.NET caching, each w3wp.exe process would have had its own independent cache state on each server. Here, we clearly see that we get the same item out of the cache but is the system really distributed (see next section…)?

Testing HA

For this test, I went to Central Administration - System - Services on Server. Then I stopped the service Distributed Cache on C7763875784. I double-checked whether this was indeed stopped at AppFabric level:

Then I simply refreshed the page on C7763875784 and could still read the cache without any problem.

However, this was working because I stopped C7763875784 not C6208759806. If I repeat this scenario but if I stop C6208759806 instead of C7763875784, then my item cannot be found in the cache anymore and is re-cached on C7763875784…

What does this mean? It means that while C6208759806 was running, it was always used whenever SPDistributedCacheWrapper.Cache.Add() was called. I decided to go one step further to find out why the hell data was not replicated accross hosts that belong to the cache cluster.

First I read, more carefully, some AppFabric documentation :). Then, I found out that at the time of writing, SharePoint does not configure high availability automatically. It means that if you want the Cache-Cluster to replicate the cached items between hosts, you need to configure it yourself. Here is how to proceed, bear in mind that I’m just sharing the results of my findings, you should of course analyze the impact of such changes on your farm but at least, you can try this approach in a sandbox and decide later whether or not it is a good idea :).

To enable HA on the SharePoint Farm, you need to:

• Ensure that your firewalls allow good AppFabric communication, see this link for more info http://msdn.microsoft.com/en-us/library/ee790914.aspx
• Open a SharePoint PowerShell session
• run Stop-CacheCluster
• run the following :

• Then run this command:

---

---

• Start-CacheCluster (or do it from CA)

From now on, any object cached with the wrapper is replicated accross hosts belonging to the cluster, giving not only your connectivity High Availability but also your cached items are Highly Available.

One step further

Locking

The AppFabric client DLLs deployed by the SharePoint setup provide two methods to handle locking. Consider the following code snippet:

---

---

Say an update method that receives an object as argument that you need to update. You can first call GetAndLock. This will try to lock the said object, in this case during 100ms. The lock handle is returned to CacheLock. The call of PutAndUnlock will update this object and explicitely unlock the object after the update is complete.

If you don’t call the PutAndUnlock method explicitely, the lock will expire after the number of milliseconds (ticks/minutes/hours…) you have defined in GetAndLock. Note that GetAndLock will throw an exception that the object is already locked out by another process/thread.

Serialization

As I stated a few times earlier, with AppFabric, the objects you put in cache are serialized. Therefore, these objects must be decorated with the Serializable attribute. Objects such as SPWeb are not serializable, therefore, you can’t cache an entire SPWeb as is.

That’s not a good idea anyway since SPWeb is also not thread-safe. AppFabric makes use of NetDataContractSerializer to serialize objects. This serializer helper makes bigger cached objects than the default BinaryFormatter used by ASP.NET. On top of that, if you’re used to work with the latter, pay attention to not encounter deserialization "conflicts".

Working with regions

AppFabric regions help represent a logical architecture that matches your application designs. In the context of SharePoint, you may want to cache User Profiles related objects in a separate region called "UserProfiles" and you may want to cache custom application data in "ApplicationDataxxx". Regions are created at runtime programmatically and have a small overhead in terms of space used (a few KG per region).

However, unless you create zillions of regions, you should not be afraid by the extra consumed space. When HA is enabled, regions are also replicated accross the cluster hosts. Regions also offer additional features such as Tagging (explained below).

It is very easy to create a region, you can just use the following line of code:

---

---

If the region already exists, it doesn’t recreate it. Another advantage I see when working with Regions is that you can reduce race conditions/multi-threading issues by storing individual objects instead of List or Dictionaries. If you have a List that’s read/updated frequently, you might quickly run into race conditions since the entire List object will be either read, or updated for every operation. If, instead, you store each invidual object with a specific key in a region, you’ll be able to update them individually, thus, reduce the number of concurrent threads that are reading/writing the same object.

Of course, that depends on the context, on the number of objects you have to deal with etc…If you’re only reading or perform CRUD. All those aspects must be taken into account before making a decision.

BulkGet

BulkGet allows developers to retrieve several cached items at once from a specific region. The following examples shows this in action:

---

---

This gives the following output:

Cache Search

You can now search for any cached item matching certain tags. This can be particularly useful since you’re free to associate any tag you’d like to your cached objects. It’s a kind of keyword association.

Reusing the same CustomType classes as earlier, here is a code sample that shows how to make use of this search mechanism:

---

---

You can associate more than one tag to a cached object. So, with a slight change to the previous example:

---

---

where I also add a second tag to all items having a "modulo 3" key compliant, I can easily make a query to get all the items that were at least tagged with CommonTag. Here is the output of the above code:

Cache Exceptions

Some specific exceptions with self-explicit names can be thrown by SharePoint components using the distributed caching. They all start with SPDistributedCache….On top of those exceptions, if you’re using the wrapper, you might encounter the regular AppFabric exceptions (similar to the SharePoint ones) + a specific exception from the wrapper which is SPDistributedCacheWrapperNoService.

This exception is raised if no server in the farm is running the Distributed Cache service.

Real World Scenario, caching User Profiles for end user search queries

A real world scenario could be that you’re storing the User Profiles within the Cache because you need to perform a lof of queries such as "autocomplete search boxes", custom advanced search centers etc…and that you don’t want to target the OOTB SharePoint search to avoid overloading it.

A while ago, I demonstrated a custom use of oData to perform such tasks, the post is available here. It would be possible to modify it to use AppFabric instead of the worker process to cache the items.

Conclusions

A lot of the topics I talked about in this article might be subject to discussions. AppFabric is now required by SharePoint 2013 but the primary purpose is to serve SharePoint…In this article, I’ve showed how we can "deviate" from the expected OOTB behavior in order to use AppFabric for custom caching. I mentioned before that I’ve not tried that approach on a real world project plus at few things I’m doing in this article might not be supported by Microsoft. These are :

• Consuming the default SharePoint cache
• Configuring the default SharePoint cache to be highly available
• Configuring the default SharePoint cache to use notification based expiration

The two last bullets are actually completely optional. Also, we are still working with the preview and might expect Microsoft to deliver a public API or a specific SharePoint guidance to deal with the underlying AppFabric layer.

Happy Coding!