14

Are there any multithreaded caching mechanisms that will work in a SQL CLR function without requiring the assembly to be registered as "unsafe"?

As also described in this post, simply using a lock statement will throw an exception on a safe assembly:

System.Security.HostProtectionException: 
Attempted to perform an operation that was forbidden by the CLR host.

The protected resources (only available with full trust) were: All
The demanded resources were: Synchronization, ExternalThreading

I want any calls to my functions to all use the same internal cache, in a thread-safe manner so that many operations can do cache reads and writes simultaneously. Essentially - I need a ConcurrentDictionary that will work in a SQLCLR "safe" assembly. Unfortunately, using ConcurrentDictionary itself gives the same exception as above.

Is there something built-in to SQLCLR or SQL Server to handle this? Or am I misunderstanding the threading model of SQLCLR?

I have read as much as I can find about the security restrictions of SQLCLR. In particular, the following articles may be useful to understand what I am talking about:

This code will ultimately be part of a library that is distributed to others, so I really don't want to be required to run it as "unsafe".

One option that I am considering (brought up in comments below by Spender) is to reach out to tempdb from within the SQLCLR code and use that as a cache instead. But I'm not quite sure exactly how to do that. I'm also not sure if it will be anywhere near as performant as an in-memory cache. See update below.

I am interested in any other alternatives that might be available. Thanks.

Example

The code below uses a static concurrent dictionary as a cache and accesses that cache via SQL CLR user-defined functions. All calls to the functions will work with the same cache. But this will not work unless the assembly is registered as "unsafe".

public class UserDefinedFunctions
{
    private static readonly ConcurrentDictionary<string,string> Cache =
                            new ConcurrentDictionary<string, string>();

    [SqlFunction]
    public static SqlString GetFromCache(string key)
    {
        string value;
        if (Cache.TryGetValue(key, out value))
            return new SqlString(value);
        return SqlString.Null;
    }

    [SqlProcedure]
    public static void AddToCache(string key, string value)
    {
        Cache.TryAdd(key, value);
    }
}

These are in an assembly called SqlClrTest, and and use the following SQL wrappers:

CREATE FUNCTION [dbo].[GetFromCache](@key nvarchar(4000))
RETURNS nvarchar(4000) WITH EXECUTE AS CALLER
AS EXTERNAL NAME [SqlClrTest].[SqlClrTest.UserDefinedFunctions].[GetFromCache]
GO

CREATE PROCEDURE [dbo].[AddToCache](@key nvarchar(4000), @value nvarchar(4000))
WITH EXECUTE AS CALLER
AS EXTERNAL NAME [SqlClrTest].[SqlClrTest.UserDefinedFunctions].[AddToCache]
GO

Then they are used in the database like this:

EXEC dbo.AddToCache 'foo', 'bar'

SELECT dbo.GetFromCache('foo')

UPDATE

I figured out how to access the database from SQLCLR using the Context Connection. The code in this Gist shows both the ConcurrentDictionary approach, and the tempdb approach. I then ran some tests, with the following results measured from client statistics (average of 10 trials):

Concurrent Dictionary Cache
10,000 Writes: 363ms
10,000 Reads :  81ms

TempDB Cache
10,000 Writes: 3546ms
10,000 Reads : 1199ms

So that throws out the idea of using a tempdb table. Is there really nothing else I can try?

Community
  • 1
  • 1
Matt Johnson-Pint
  • 197,368
  • 66
  • 382
  • 508
  • It is not entirely clear what you are asking her 9although this could just be me :]). What code is throwing this exception and what is it you are trying to do? – MoonKnight Apr 30 '13 at 15:30
  • 1
    @Killercam - Added code example. – Matt Johnson-Pint Apr 30 '13 at 15:37
  • 3
    Perhaps a better place to make a cache is in the database itself as a table? You'll get synchronisation for free then. – spender Apr 30 '13 at 15:47
  • @spender - I think you might be on to something. Can I manage my own tempdb data from within a SQLCLR function? Any idea if it will perform as well as an in-memory cache? If you can show an answer with a code example, that would be awesome. – Matt Johnson-Pint Apr 30 '13 at 15:50
  • While your question is valid here, SQL-CLR questions typically get a much better response at dba.stackexchange.com. – RBarryYoung Apr 30 '13 at 15:50
  • Sorry @MattJohnson, I've got nothing. SQL-CLR is an alien world to me. – spender Apr 30 '13 at 15:51
  • @RBarryYoung - I checked dba.se, but there are only 9 questions tagged with `sql-clr` there, and 376 here. Of those that I found there, even the ones not tagged as such, there were only a few with SQL code, and I didn't see any with C# code at all. I will need to find a different way to ask the question if I post there. – Matt Johnson-Pint Apr 30 '13 at 16:27
  • @MattJohnson You are correct about the numbers, but you shouldn't need to rephrase the question. There are probably only a handful of people on either either site who are able to answer a question like this, but those who are are *extremely* qualified (for example: Paul White) and will have no trouble with the C#, etc. Since that is a much smaller site, questions of a relatively obscure (but on-topic) expertise like this tend to get more immediate (and higher-level) response. You can also wait and see if you get what you want here, and if not, then try over there. – RBarryYoung Apr 30 '13 at 16:36
  • @RBarryYoung - Ok. Thanks for the friendly advice. I have also updated the question with some more details. – Matt Johnson-Pint Apr 30 '13 at 16:38
  • I'm voting for @spender's comment - sounds like you need a table with an index. I know it might sound deliberately unhelpful, but it's almost impossible to achieve what you're looking for in .Net without thread synchronisation primitives, which require the unsafe context – Andras Zoltan Apr 30 '13 at 16:39
  • Have you tried a pure t-sql approach using tempdb? t-sql to clr to t-sql isn't going to offer any advantages over a pure t-sql approach. Remember that clr in sql server is optimized for returning data from clr to sql, not for pulling data across the context connection. – StrayCatDBA May 01 '13 at 01:32
  • Also, if I'm reading your numbers correctly, the tempdb approach above gives .35 ms for writes and .12 ms for reads. Are you sure that isn't fast enough? What amount of overhead is that per call? – StrayCatDBA May 01 '13 at 01:42
  • @StrayCatDBA - There is a lot more to my code that would not work well in pure SQL. SQLCLR fits very well, I just don't have the caching ability I'd have in any other .Net library. The data being cached is used internally by the rest of the code in the library - not externally. It needs fast access because it will hold thousands of data points that it will do calculations over in bulk. I'm really just looking for a threadsafe, in-memory cache that I can use within my own SQLCLR code. – Matt Johnson-Pint May 01 '13 at 02:51
  • What happens if sql unloads your app domain and you lose the cache? Sql will do that under memory pressure. – StrayCatDBA May 01 '13 at 03:19
  • 1
    @StrayCatDBA Then I get a cache miss and the next load takes a bit longer while the operation performs again, but then it's cached again until the next unload. When you're going to do something 1000 times, do it once and cache it. If the cache is gone when you come back so you have to do it again, that's still 999 times you can use the cache. Most caching works on this principle. – Matt Johnson-Pint May 01 '13 at 04:47
  • @RBarryYoung et al, I know this question is approaching 2 years old, but you might want to take a look-see at the answer I just posted. Better late than never, right? ;-) – Solomon Rutzky Jan 27 '15 at 22:16

5 Answers5

5

I've added a comment that says something similar, but I'm going to put it here as an answer instead, because I think it might need some background.

ConcurrentDictionary, as you've correctly pointed out, requires UNSAFE ultimately because it uses thread synchronisation primitives beyond even lock - this explicitly requires access to lower-level OS resources, and therefore requires the code fishing outside of the SQL hosting environment.

So the only way you can get a solution that doesn't require UNSAFE, is to use one which doesn't use any locks or other thread synchronisation primitives. However, if the underlying structure is a .Net Dictionary then the only truly safe way to share it across multiple threads is to use Lock or an Interlocked.CompareExchange (see here) with a spin wait. I can't seem to find any information on whether the latter is allowed under the SAFE permission set, but my guess is that it's not.

I'd also be questioning the validity of applying a CLR-based solution to this problem inside a database engine, whose indexing-and-lookup capability is likely to be far in excess of any hosted CLR solution.

Andras Zoltan
  • 40,853
  • 11
  • 98
  • 156
  • I attempted a more database-centric solution, using tempdb as the cache. It works in a "safe" assembly, but performance is 10 to 15 times slower. See the update in my question. And thanks. – Matt Johnson-Pint Apr 30 '13 at 18:31
  • 3
    One point that hasn't been brought up is why sql disallows these locking primitive in safe assemblies. The reason is the cooperative scheduling that sqlos uses. Tasks in sql server are expected to yield periodically. If a thread is running clr code and waits on a resource without yielding, then all sorts of issue can occur. Even if a .net lock does cause the thread to yield, it may incur a 4ms wait until it is scheduled again. The interlocked exchange would be your best bet, but may force the unsafe assembly. – StrayCatDBA May 01 '13 at 03:13
  • @MattJohnson for your solution that uses a table, why did you continue to do it through CLR functions? A better test of that approach would have been to have written straightforward SQL functions/procedures - ADO.Net Context Connections are going to be much slower. – Andras Zoltan May 01 '13 at 07:57
  • just looking at your comments on the question, it seems you need this cache for SQLCLR code, not necessarily for database operations (i.e. other stored procedures and functions) - in which case it's actually unlikely that you'll get better performance... If your were looking to use it from pure SQL then the table solution would be better. – Andras Zoltan May 01 '13 at 13:28
  • Marking this as the answer because it pretty much says "you can't" - which turns out to be true. Thanks very much for trying though! :) – Matt Johnson-Pint May 19 '13 at 03:45
  • Not sure "you can't" is the correct conclusion. Have you looked into lock-free concurrent dictionaries such as: http://www.cse.chalmers.se/~tsigas/papers/Lock-Free_Dictionary.pdf – Triynko Nov 04 '13 at 23:06
  • 1
    I have implemented his lock-free queue/stack in C++ and then tried doing the same in C#; it's not possible to do it effectively (i.e. it still works, but not better than simply using a lock) because you don't have access to the low-level CAS functions in .Net. Yes, you do have Interlocked.CompareExchange, but not double-word, and it's just a wrapper for the OS function. While also the versioned pointer is nigh impossible to implement properly in .Net in such a way that it can be treated as a number for the CAS function. In any case, even if it were - it'll still need to be `UNSAFE`. – Andras Zoltan Nov 05 '13 at 09:24
  • So is Interlocked.CompareExchange allowed in a SAFE assembly? If so, then for a caching mechanism it would probably be sufficient to create lock-free dictionary, even if updates are non-optimal. For example, by creating a static readonly reference to an object which implements basic dictionary operations and stores an instance variable that is an immutable dictionary, read operations from the dictionary will be thread-safe. – Triynko Nov 06 '13 at 06:53
  • Write operations will simply involve grabbing a reference to the dictionary, creating a working copy of the dictionary, updating the working copy, and doing a compare exchange against the original dictionary. If the return value of the compare exchange is not the same as the value you based your copy on, then you know the exchange failed and the dictionary has been updated by another thread since you copied it, and you simply retry the change. Although it's non-optimal and may use more memory during updates, it's sufficient and simple for a cache, and doesn't impair read performance. – Triynko Nov 06 '13 at 06:55
  • 1
    @Triynko and Andras, `Interlocked.CompareExchange` is allowed in a `SAFE` assembly, but it is effectively useless as it needs to operate on shared variable. _That_ is not going to happen in a `SAFE` assembly: either the variable has to be static (not gonna happen) OR the threads that will act upon the instance-level variable need to be launched from that instance of that class, but that also can't happen as all SQLCLR methods are `static`. Please see my answer for both a) how to accomplish this using `SAFE` (even if it might be risky) and b) an alternate proposal. – Solomon Rutzky Jan 27 '15 at 22:25
5

The accepted answer is not correct. Interlocked.CompareExchange is not an option since it requires a shared resource to update, and there is no way to create said static variable, in a SAFE Assembly, that can be updated.

There is (for the most part) no way to cache data across calls in a SAFE Assembly (nor should there be). The reason is that there is a single instance of the class (well, within the App Domain which is per-database per-owner) that is shared across all sessions. That behavior is, more often than not, highly undesirable.

However, I did say "for the most part" it was not possible. There is a way, though I am not sure if it is a bug or intended to be this way. I would err on the side of it being a bug since again, sharing a variable across sessions is a very precarious activity. Nonetheless, you can (do so at your own risk, AND this is not specifically thread safe, but might still work) modify static readonly collections. Yup. As in:

using Microsoft.SqlServer.Server;
using System.Data.SqlTypes;
using System.Collections;

public class CachingStuff
{
    private static readonly Hashtable _KeyValuePairs = new Hashtable();

    [SqlFunction(DataAccess = DataAccessKind.None, IsDeterministic = true)]
    public static SqlString GetKVP(SqlString KeyToGet)
    {
        if (_KeyValuePairs.ContainsKey(KeyToGet.Value))
        {
            return _KeyValuePairs[KeyToGet.Value].ToString();
        }

        return SqlString.Null;
    }

    [SqlProcedure]
    public static void SetKVP(SqlString KeyToSet, SqlString ValueToSet)
    {
        if (!_KeyValuePairs.ContainsKey(KeyToSet.Value))
        {
            _KeyValuePairs.Add(KeyToSet.Value, ValueToSet.Value);
        }

        return;
    }

    [SqlProcedure]
    public static void UnsetKVP(SqlString KeyToUnset)
    {
        _KeyValuePairs.Remove(KeyToUnset.Value);
        return;
    }
}

And running the above, with the database set as TRUSTWORTHY OFF and the assembly set to SAFE, we get:

EXEC dbo.SetKVP 'f', 'sdfdg';

SELECT dbo.GetKVP('f'); -- sdfdg

SELECT dbo.GetKVP('g'); -- NULL

EXEC dbo.UnsetKVP 'f';

SELECT dbo.GetKVP('f'); -- NULL

That all being said, there is probably a better way that is not SAFE but also not UNSAFE. Since the desire is to use memory for caching of repeatedly used values, why not set up a memcached or redis server and create SQLCLR functions to communicate with it? That would only require setting the assembly to EXTERNAL_ACCESS.

This way you don't have to worry about several issues:

  • consuming a bunch of memory which could/should be used for queries.

  • there is no automatic expiration of the data held in static variables. It exists until you remove it or the App Domain gets unloaded, which might not happen for a long time. But memcached and redis do allow for setting an expiration time.

  • this is not explicitly thread safe. But cache servers are.

Solomon Rutzky
  • 41,664
  • 6
  • 112
  • 149
  • 1
    I think this may be exactly what I was looking for. I'll try out this approach and let you know how well it works for the scenario I was intending it for. If all goes well, I'll change this to the accepted answer. Thanks! – Matt Johnson-Pint Jan 28 '15 at 00:47
  • @MattJohnson I updated to make a note regarding thread safety since you mentioned it in the question. May I ask what the scenario is? I am curious why setting the Assembly to `UNSAFE` is being avoided, outside of mere preference (which I certainly understand). – Solomon Rutzky Jan 28 '15 at 05:16
  • Actually, it turned-out it is possible to use Interlocked.CompareExchange after all. You can just declare a static readonly field of _reference_ type, containing an int field, and then use that field for interlocked operations. – Branko Dimitrijevic Nov 21 '15 at 17:56
  • As you said, this approach isn't thread safe, which makes it unsuitable for this purpose. Hashtable does have a `Synchronized` wrapper, but that has its own set of locking, which I'm pretty sure won't work under SQLCLR's "SAFE" mode (though I haven't tried yet). Still the limitations described [in the MSDN](https://msdn.microsoft.com/en-us/library/system.collections.hashtable.aspx#Anchor_10) seem like they would be a problem. – Matt Johnson-Pint Feb 02 '16 at 17:55
  • I do agree that using a different cache server would be an option in a distributed solution, but in this particular case I was looking for an answer specifically for a library targeting SQLCLR, scoped to a single server. – Matt Johnson-Pint Feb 02 '16 at 17:57
  • @MattJohnson Hey there. Thanks for addressing those points. I will say that I recommended the cache server before learning of the intended purpose. And, I have an idea to make this thread safe and without much code, just need time to test it. Still, knowing more about the project, I am wondering if you even need this in the first place. When you have a moment, we should discuss certain details of the project. It could be that translating this part so that it behaves the same but works in SQLCLR might not be what is best for the project. The SQLCLR environment sometimes calls for a new model. – Solomon Rutzky Feb 02 '16 at 19:37
1

SQL Server locking functions sp_getapplock and sp_releaseapplock can be used in SAFE context. Employ them to protect an ordinary Dictionary and you have yourself a cache!

The price of locking this way is much worse than ordinary lock, but that may not be an issue if you are accessing your cache in a relatively coarsely-grained way.

--- UPDATE ---

The Interlocked.CompareExchange can be used on a field contained in a static instance. The static reference can be made readonly, but a field in the referenced object can still be mutable, and therefore usable by Interlocked.CompareExchange.

Both Interlocked.CompareExchange and static readonly are allowed in SAFE context. Performance is much better than sp_getapplock.

Branko Dimitrijevic
  • 47,349
  • 10
  • 80
  • 152
1

Based on Andras answer, here is my implantation of a "SharedCache" to read and write in a dictionary in SAFE permission.

EvalManager (Static)

using System;
using System.Collections.Generic;
using Z.Expressions.SqlServer.Eval;

namespace Z.Expressions
{
    /// <summary>Manager class for eval.</summary>
    public static class EvalManager
    {
        /// <summary>The cache for EvalDelegate.</summary>
        public static readonly SharedCache<string, EvalDelegate> CacheDelegate = new SharedCache<string, EvalDelegate>();

        /// <summary>The cache for SQLNETItem.</summary>
        public static readonly SharedCache<string, SQLNETItem> CacheItem = new SharedCache<string, SQLNETItem>();

        /// <summary>The shared lock.</summary>
        public static readonly SharedLock SharedLock;

        static EvalManager()
        {
            // ENSURE to create lock first
            SharedLock = new SharedLock();
        }
    }
}

SharedLock

using System.Threading;

namespace Z.Expressions.SqlServer.Eval
{
    /// <summary>A shared lock.</summary>
    public class SharedLock
    {
        /// <summary>Acquires the lock on the specified lockValue.</summary>
        /// <param name="lockValue">[in,out] The lock value.</param>
        public static void AcquireLock(ref int lockValue)
        {
            do
            {
                // TODO: it's possible to wait 10 ticks? Thread.Sleep doesn't really support it.
            } while (0 != Interlocked.CompareExchange(ref lockValue, 1, 0));
        }

        /// <summary>Releases the lock on the specified lockValue.</summary>
        /// <param name="lockValue">[in,out] The lock value.</param>
        public static void ReleaseLock(ref int lockValue)
        {
            Interlocked.CompareExchange(ref lockValue, 0, 1);
        }

        /// <summary>Attempts to acquire lock on the specified lockvalue.</summary>
        /// <param name="lockValue">[in,out] The lock value.</param>
        /// <returns>true if it succeeds, false if it fails.</returns>
        public static bool TryAcquireLock(ref int lockValue)
        {
            return 0 == Interlocked.CompareExchange(ref lockValue, 1, 0);
        }
    }
}

SharedCache

using System;
using System.Collections.Generic;

namespace Z.Expressions.SqlServer.Eval
{
    /// <summary>A shared cache.</summary>
    /// <typeparam name="TKey">Type of key.</typeparam>
    /// <typeparam name="TValue">Type of value.</typeparam>
    public class SharedCache<TKey, TValue>
    {
        /// <summary>The lock value.</summary>
        public int LockValue;

        /// <summary>Default constructor.</summary>
        public SharedCache()
        {
            InnerDictionary = new Dictionary<TKey, TValue>();
        }

        /// <summary>Gets the number of items cached.</summary>
        /// <value>The number of items cached.</value>
        public int Count
        {
            get { return InnerDictionary.Count; }
        }

        /// <summary>Gets or sets the inner dictionary used to cache items.</summary>
        /// <value>The inner dictionary used to cache items.</value>
        public Dictionary<TKey, TValue> InnerDictionary { get; set; }

        /// <summary>Acquires the lock on the shared cache.</summary>
        public void AcquireLock()
        {
            SharedLock.AcquireLock(ref LockValue);
        }

        /// <summary>Adds or updates a cache value for the specified key.</summary>
        /// <param name="key">The cache key.</param>
        /// <param name="value">The cache value used to add.</param>
        /// <param name="updateValueFactory">The cache value factory used to update.</param>
        /// <returns>The value added or updated in the cache for the specified key.</returns>
        public TValue AddOrUpdate(TKey key, TValue value, Func<TKey, TValue, TValue> updateValueFactory)
        {
            try
            {
                AcquireLock();

                TValue oldValue;
                if (InnerDictionary.TryGetValue(key, out oldValue))
                {
                    value = updateValueFactory(key, oldValue);
                    InnerDictionary[key] = value;
                }
                else
                {
                    InnerDictionary.Add(key, value);
                }

                return value;
            }
            finally
            {
                ReleaseLock();
            }
        }

        /// <summary>Adds or update a cache value for the specified key.</summary>
        /// <param name="key">The cache key.</param>
        /// <param name="addValueFactory">The cache value factory used to add.</param>
        /// <param name="updateValueFactory">The cache value factory used to update.</param>
        /// <returns>The value added or updated in the cache for the specified key.</returns>
        public TValue AddOrUpdate(TKey key, Func<TKey, TValue> addValueFactory, Func<TKey, TValue, TValue> updateValueFactory)
        {
            try
            {
                AcquireLock();

                TValue value;
                TValue oldValue;

                if (InnerDictionary.TryGetValue(key, out oldValue))
                {
                    value = updateValueFactory(key, oldValue);
                    InnerDictionary[key] = value;
                }
                else
                {
                    value = addValueFactory(key);
                    InnerDictionary.Add(key, value);
                }


                return value;
            }
            finally
            {
                ReleaseLock();
            }
        }

        /// <summary>Clears all cached items.</summary>
        public void Clear()
        {
            try
            {
                AcquireLock();
                InnerDictionary.Clear();
            }
            finally
            {
                ReleaseLock();
            }
        }


        /// <summary>Releases the lock on the shared cache.</summary>
        public void ReleaseLock()
        {
            SharedLock.ReleaseLock(ref LockValue);
        }

        /// <summary>Attempts to add a value in the shared cache for the specified key.</summary>
        /// <param name="key">The key.</param>
        /// <param name="value">The value.</param>
        /// <returns>true if it succeeds, false if it fails.</returns>
        public bool TryAdd(TKey key, TValue value)
        {
            try
            {
                AcquireLock();

                if (!InnerDictionary.ContainsKey(key))
                {
                    InnerDictionary.Add(key, value);
                }

                return true;
            }
            finally
            {
                ReleaseLock();
            }
        }

        /// <summary>Attempts to remove a key from the shared cache.</summary>
        /// <param name="key">The key.</param>
        /// <param name="value">[out] The value.</param>
        /// <returns>true if it succeeds, false if it fails.</returns>
        public bool TryRemove(TKey key, out TValue value)
        {
            try
            {
                AcquireLock();

                var isRemoved = InnerDictionary.TryGetValue(key, out value);
                if (isRemoved)
                {
                    InnerDictionary.Remove(key);
                }

                return isRemoved;
            }
            finally
            {
                ReleaseLock();
            }
        }

        /// <summary>Attempts to get value from the shared cache for the specified key.</summary>
        /// <param name="key">The key.</param>
        /// <param name="value">[out] The value.</param>
        /// <returns>true if it succeeds, false if it fails.</returns>
        public bool TryGetValue(TKey key, out TValue value)
        {
            try
            {
                return InnerDictionary.TryGetValue(key, out value);
            }
            catch (Exception)
            {
                value = default(TValue);
                return false;
            }
        }
    }
}

Source Files:

Jonathan Magnan
  • 9,283
  • 2
  • 28
  • 53
0

Would your needs be satisfied with a table variable? They're kept in memory, as long as possible anyway, so performance should be excellent. Not so useful if you need to maintain your cache between app calls, of course.

Created as a type, you can also pass such a table into a sproc or UDF.

  • 1
    The data I need to cache would have to survive across threads and processes. If you can show me how to hold on to a table variable with global scope, you'd be my new hero. :) – Matt Johnson-Pint Apr 30 '13 at 23:21
  • @MattJohnson and Jon, even if there was some sort of global table variable, you wouldn't have saved much over the Temp Table since table variables are persisted in tempdb; they are lighter weight than temp tables, but not 100% memory. Please see my answer for how to do this (even if risky) as well as an alternate proposal :). – Solomon Rutzky Jan 27 '15 at 22:28