1

I am using entity framework 5 for a query like this:

var query = 
    from i in context.Instrument
    from p in i.InstrumentPerformance  // 1 : n
    where p.PortfolioScenarioID == 6013
    select i;

I want to store a queryable respresentation of this (filtered) query in memory. Ideally, I would be able to disconnect the context and still request a specific InstrumentPerformance collection like so:

var perf = query.First(i => i.InstrumentID == 407240).InstrumentPerformance;

But this - of course - does not produce the desired result, since the "perf" object will contain an InstrumentPerformance collection that contains every 1:n joined InstrumentPerformance entity (whether its PortfolioScenarioID is 6013 or not) and it will retrieve these entities via lazy loading, with context.ContextOptions.LazyLoadingEnabled = false (or the context runnning out of scope) the query will not yield anyting.

So this is far from where I want to get: an easy to query in-memory representation from the original query. I tried to materialize into dictionaries and similar approaches, but ended up coding custom data objects for the result which I would like to avoid.

So my question is: what is the recommended method to get such in-memory view?

EDIT: I am currently using two dictionaries to cache the data, e.g:

var instruments = (
    from i in context.Instrument
    from p in i.InstrumentPerformance
    where p.PortfolioScenarioID == 6013
    select i)
    .ToDictionary (i => p.InstrumentID, i => i);

var performances = (
    from i in context.Instrument
    from p in i.InstrumentPerformance
    where p.PortfolioScenarioID == 6013
    select p)
    .ToDictionary (p => p.InstrumentID, p => p);

However, this requires two roundtrips to the database where one seems sufficient and more importantly the semantics for querying the performance data (which is now performances[InstrumentID]) is inconsistent with the EF way of querying (which should be instrument.InstrumentPerformance.First() and the like).

TvdH
  • 896
  • 10
  • 27

2 Answers2

1

It is possible to retrieve the objects in one take first and then create the dictionaries by:

var query = 
    (from i in context.Instrument
    select new { 
                 i,
                 ps = i.InstrumentPerformance
                          .Where(p.PortfolioScenarioID == 6013)
               }).AsEnumerable()
               .Select(x => x.i);

This materializes and selects Instrument entities and, here's the trick, their partly loaded InstrumentPerformance collections. I.e. the instruments only contain InstrumentPerformance entities that meet the condition PortfolioScenarioID == 6013. This is because EF runs a process known as relationship fixup that ties child objects to the right parent object when they are fetched from the database.

So now you can dispose the context and any time after that do

var perf = query.First(i => i.InstrumentID == 407240).InstrumentPerformance;

or build your dictionaries using from i in query in stead of from i in context.Instrument.

IMPORTANT: lazy loading should be disabled, otherwise EF will still try to load the full collections when they are addressed.

Gert Arnold
  • 93,904
  • 24
  • 179
  • 256
  • I tried `.Where(pf => pf.PortfolioScenarioID == 6013)` and `.Where(pf => p.PortfolioScenarioID == 6013)` but in both cases the sql produces out of memory exceptions. Maybe just a minor change is required to get this promising approach to work? I can not figure out. – TvdH Jun 10 '13 at 10:27
  • Is the stack trace still accessible and can you indicate where the memory exception is thrown? I can't figure why this should happen, I've got similar queries running OK. – Gert Arnold Jun 10 '13 at 10:34
  • It happens when the query is being executed on the server. I can execute the query in Mngmt Studio and will also get an out of memory error because InstrumentPerformance is joined with itself on InstrumentID - returning way too many records. Maybe the `.Where(bool)` condition in your example is ment to be this way and I cannot use it because I am missing an extension? – TvdH Jun 10 '13 at 12:58
  • Isn't that a flaw in the associations? I think `InstrumentPerformance` should have an FK to `Instrument` (`InstrumentID`). I don't see how you can join its `InstrumentID` with itself. Maybe show the data model to clarify. – Gert Arnold Jun 10 '13 at 13:06
  • Yes, Instrument / InstrumentPerformance are joined 1:n on InstrumentID (and this assosciation seems OK in the model, i.e. `Instrument.First().InstrumentPerformance` only returns the InstrumentPerformance data for the first Instrument). This will hopefully illustrate: [link](http://sdrv.ms/11Cxs0W) – TvdH Jun 10 '13 at 13:35
  • Ahh... small glitch in my query. It's even simpler :) – Gert Arnold Jun 10 '13 at 13:44
  • thank you, works great now (still need `.Where(p => p.PortfolioScenarioID == 6013` to run it in my environment) – TvdH Jun 10 '13 at 15:16
0

Look at EntityCollection<T> CreateSourceQuery and Attach. I think you could do this (not tested):

var instrumentQuery = 
    from i in context.Instrument
    from p in i.InstrumentPerformance  // 1 : n
    where p.PortfolioScenarioID == 6013
    select i;
var instruments = instrumentQuery.ToList();
foreach (var instrument in instruments) {
    var performanceQuery =
        instrument.InstrumentPerformance.CreateSourceQuery()
            .Where(p => p.PortfolioScenarioID == 6013);
    instrument.InstrumentPerformance.Attach(performanceQuery);
}

This executes everything at once (no lazy loading) and has a bit of code duplication, but it would result in a list of Instrument where each i.InstrumentPerformance returns the filtered collection, meaning any subsequent code that operates on it can treat it like any other EF collection without needing to know the details of the query.

nmclean
  • 7,266
  • 2
  • 24
  • 37
  • I tried this interesting appoach. The "foreach" is a problem because it produces 10s of thousands DB roundtrips and for some reason the p.PortfolioScenarioID == 6013 filter does not work. It looks like the data has been materialized and I can disconnect the context and still access instrument.First().InstrumentPerformance so this is good, but the omitted where clause and the foreach loop currently make this impractical for my application. – TvdH Jun 09 '13 at 22:19
  • @TvdH I don't know why the filter didn't work... you may need to call `Execute` and pass the result of that to `Attach` instead. Anyway, you're right, this is no good if you need to load thousands at once. Gert's solution is probably the right one for you. What I have used this for personally is recursive relationships (which isn't really possible to do with a single query) and I was loading them on-demand rather than up-front. – nmclean Jun 10 '13 at 01:56