Today we will talk about a problem that can arise when uploading documents in a library => slowness!
When I say “we”, I mean my colleague Christopher Clement and myself. Having faced this problem together, we figured we would write together about it!
In the case we studied, the upload of a document in an empty library took 200ms whereas the same document in another library of the same site took 18 seconds!
The differences between the two libraries
- No documents
- No content types added
- 32136 items
- 72 content types including 70 derived from the “Document Set” content type
Wait, WHAT??!! 30000 items?? That’s the reason, it is poorly indexed, and so on, and so on…
In fact, no, that wasn’t the cause of the issue, the indexing and structure were correct, so let’s move on to the other difference, the number of content types.
Actually, this library uses a significant number of content types inheriting from Document Set.
As a reminder, a Document Set is a super folder that brings interesting features such as the configuration of a custom page and “shared metadata”.
This second feature is the reason why Document Sets are used here, it allows to share or rather propagate its own metadata to all the documents that it contains. A feature that we had used several times on previous projects. The difference here is the large number of different content types.
Note : If the problem we have encountered has occurred on an on-premise environment, you should know that we performed the tests on the on-premise AND on an Office 365 environment with the same results.
Our test scenario was to upload a “test.txt” document containing only the word “test” in libraries with different configurations. On our production library, we have the following configuration:
- 2643 document sets
- 32136 items in total (including documents sets)
- 72 content types
The upload on this library of our file “test.txt” takes between 5 and 20 seconds to respond, depending on the solicitation of the resources on the server. In our tests, the response time was 18 seconds.
So we started by measuring the response time for the upload of the same document on a brand new library without any specific configuration.
The response time (0.87 seconds) is minimal compared to our production library.
The next step was to gradually add content types (inheriting from Document Set) to measure the impact for different numbers of Content Types added. We’re talking about the content types that are configured on the list, not the “documents sets” items added to the library. Here are the results:
For 1 content type inheriting from Document Set: 312 msFor 30 content types inheriting from Document Set: 4.93sFor 50 content types inheriting from Document Set: 11.18s
We clearly observe that the number of Document Set content types available in the list impacts the time required for the upload of a document.
We have already noticed in the past that the functionality of a Document Set is provided by Event Receivers added to the list, there are actually 4: ItemAdding, ItemAdded, ItemUpdating and ItemUpdated.
So we checked on that side if that could be the reason. We were astonished at what we found out.
283 Event Receivers are attached to the library !
- 283 Event Receivers (285 – 2 header lines)
- They are ALL in Synchronous mode, even those in -ed which are not by default.
We could have thought that Event Receivers would have been added once for all DocumentSet content types, but no! For each Content Type added, 4 Event Receivers are added to the list.
Synchronous events receivers are executed before the response is returned to the client. In our case, at least 140 event receivers are executed each time an item is added, and another 140 are executed at each update.
Moreover, if you ever had the opportunity to develop a custom (server side) event receiver, you might have realized that you configure it through its type (.NET Class) fully qualified name. That means, somehow, it is dynamically (and probably lazily) loaded. It must have its footprint has well on the performance.
This explains the slowness.
Solving this problem can be done in 2 ways according to your needs
1 : Reducing the number of content types
If the number of content types inheriting from “Document Set” may decrease, reduce it to the maximum, a unique “Document Set” content type would be ideal. Migrate your existing documents to this unique content type.
If you need to migrate content types from existing documents, I invite you to read these posts which explain a way to do it: (https://ypcode.wordpress.com/2016/12/08/cross-sharepoint-platform-maintenance-tools-using-pnp-powershell/) (https://ypcode.wordpress.com/2017/07/22/powershell-quicktip-3-characters-to-improve-a-script/)
Then delete content types inheriting from “Document Set” that are no longer used in the library. This will remove related event receivers.
2 : Division of the structure into several libraries
If you need to keep several types of “Document Set” content type, we recommend that you divide the content between several libraries, each using one of your content types.
Resolution bis (on-premise only)
We also noticed that the auto-increment of the database was set to 1 mega, we increased this size to reduce SQL work when uploading document.
Hoping this helps!
Christopher and Yannick