As I showed in my first post, I also have my Activities entity – and from reading (way) too much through the various literature on how to make SharePoint scale with large lists, I see that one of the recommended ways of scaling is to distribute your data into multiple lists. Also with my last post in mind, about throttling, I figure, that keeping many relatively small lists would be better, that one big potentially unreadable list.
From SharePoint 2010 Microsoft promises us, that they now are able to handle large list – namely because they now have implemented “Query trottling” that prohibits (limits?) queries, that return too many items. So you think to yourself: Wow – that’s nice! Now I can implement applications that supports large amounts of data; You start implementing your solution; You test it on your development box (where you probably like me are running as Farm Admin, so you can install your new application) – you even do a test with at least 5001 items in a list (over your throttle limit), and everything seems to be working. Then you hand it over to the customer, for them to install, and it works for a couple of days – but as soon as the customer inserts item number 5001, everything stops to work.
From working extensively with SharePoint, and more often than not, with problems in SharePoint where being a SharePoint consultant becomes a bit, shall we say: *problematic*, I want to share some of the experiences and ideas that I have made along the way. I’ll start out very simple with something about SharePoint that has been annoying me for some time, and has been very annoying for my colleagues and development partners – namely searching for items through CAML.