Archive for November 11th, 2010

Some LINQ Performance “Rules of thumb”


Use indexed classes

Use indexed classes, or more precisely use structures which support keyed access (Dictionary Class and Lookup Class). The SQL analogy of these structures being like indexed tables, in comparison to List which behaves like an unindexed SQL table, seems to hold true with LINQ.

There is a tipping point with both SQL and LINQ, where the cost of establishing the indexed storage is less than the performance improvement you gain from using it. With LINQ you will have to find that through experimentation with your own use. Use the Stopwatch Class and measure the impact of one strategy over the other.

The performance impact of using indexed data structures is particularly marked when you doing joins (equijoin) between sequences in LINQ think seriously about feeding the sequences into a structure which supports a key, and then joining on the keys.

Use the Join Syntax When Performing an Equijoin

Use the join syntax in LINQ. By inspections/divination this  syntax

var seed = new List<XElement>();
var seq1 = seed.Select(A => new { key = A.Name, value = A })
    .ToLookup(A => A.key, A => A.value);
var seq2 = seed.Select(A => new { key = A.Name, value = A })
    .ToLookup(A => A.key, A => A.value);
var example1 = from a in seq1
               join b in seq2 on a.Key equals b.Key
               select a;

is “better” (can be significantly faster) than syntax

var seed = new List<XElement>();
var seq1 = seed.Select(A => new { key = A.Name, value = A })
    .ToLookup(A => A.key, A => A.value);
var seq2 = seed.Select(A => new { key = A.Name, value = A })
    .ToLookup(A => A.key, A => A.value);
var example2 = from a in seq1
               from b in seq2
               where a.Key == b.Key
               select a;

I am not sure what is happening under the hood, but the execution elapse times are far better when using the join syntax. Part of the improvement probably comes from the fact that in the second version the string “==” operator is firing multiple time (that’s what the Visual Studio Profiler said).

Beware of Implicit Constructors Firing Multiple Times

This one is another which the Visual Studio Profiler highlighted. The following syntax will fire the XName constructor multiple times. The

var seed = new List<XElement>();
// NB, both == and the Attribute
var example3 = from a in seed
               where a.Name == "Fred" && a.Attribute("Fred").Value == "Value"
               select a;

The following is a very simple optimisation, but one which saves “a bucket load” of CPU cycles (multiple calls to the XName constructor).

var seed = new List<XElement>();
// Form the XName once, and use it multiple time
XName Fred = "Fred";
var example3 = from a in seed
               where a.Name == Fred && a.Attribute(Fred).Value == "Value"
               select a;

Beware of || in Where Clauses

This one comes straight from my experience with SQL (both Oracle and SQL Server), and translates into what I’ve observed in the execution of LINQ statements. The or operator (||) in LINQ where clauses can cause very slow execution. .

For simple cases you can get a performance boost just by rewriting the LINQ statement as a Union between the two different sides of the or. I’ll  leave it to you to verify the result you get out are the same. Also, this is another optimisation which using the Stopwatch Class to test the performance of a before and after case. For example:

var seed = new List<XElement>();
XName Fred = "Fred";
XName Joe = "Joe";
var example4 = from a in seed
               where a.Attribute(Fred).Value == "test1" || a.Attribute(Joe).Value == "test2"
               select a;

could be rewritten as:

var example5 = (from a in seed
                where a.Attribute(Fred).Value == "test1"
                select a)
                    .Union(from a in seed
                           where a.Attribute(Joe).Value == "test2"
                           select a);

Conclusion

That’s 4 lessons I’ve learned in the last couple of days performance tuning my  DGML generating and manipulating program. The Stopwatch Class and Visual Studio Profiler have been invaluable in identifying the performance bottlenecks, or hot spots, in the system. The judicious redesign of some of the sequences to use Dictionary and Lookup classes, and joining the sequences on the keys of those collections, yielded big improvements in the performance.

LINQ seems to be “balanced on a knife edge” at time. Minor redesigns can result in the execution time going from minutes to seconds in execution time.

, , , , , , , , , , , , ,

Leave a comment

Toasted Muesli Recipe


IngredientsMuesli 002

Method

  1. Preheat oven to 150° C
  2. Line a deep baking dish with baking paper.
  3. Mix and melt the honey, brown sugar and olive oil in a saucepan.
  4. Mix the remaining ingredients.
  5. Add the honey, oil and sugar mixture, mixing well.
  6. Pour into lined baking dish, and bake for 25 to 30 minutes, stirring occasionally (every 10 minutes or so).
  7. Remove from the oven and allow to cool. It will set into a clumpy mass. When cooled, break up the clumps and store in an air tight container.

Recipe Notes

  1. I’ve used Light Olive Oil. I did not want the full flavour of a Extra Virgin Olive Oil.
  2. I’ve used Grey Gum Honey in this batch. It is a lighter and softer honey.
  3. I’d be tempted to lightly toast the sesame seeds first. Just put them into a non-stick frying pan over a low heat, toast until there is just a touch of colour. I’ve the feeling that the sesame seeds are getting a bit lost (flavour wise)in the final product.
  4. Brown Sugar, is the soft brown variety. You could probably substitute an equal volume of golden syrup.
  5. 3 Cups of Oats may seem a bit excessive, but in the final muesli it does not seem to dominate.

Conclusions

In a word delicious. I had to put it into the storage containers, otherwise I  would have kept picking at it.

Like all recipes, feel free to add or subtract as the mood takes you.

Some ideas for variations based on some geographical themes:

  • Tropical flavour (building on the coconut flavour already there), try adding dried mango, dried pineapple and banana chips. I’d probably not put these through the oven, I’ve no idea if they would burn and ruin the batch. I’d just add these  after the mixture has cooled after baking.
  • European winter feel, try adding dried pear, dried cherries and walnut pieces. Replacing the Soft Brown Sugar with an equal quantity of Golden Syrup, may also add to the European winter flavours.
  • Middle Eastern feel try dried figs, dried apricots, sultanas and raisins.  Also, I’d add a teaspoon of cinnamon into  this as well.

, , , , , , , ,

2 Comments

%d bloggers like this: