Posts Tagged Parallel computing

Where are the Visual Studio Parallel Debug Tools?


If you are just venturing into doing things in parallel with Visual Studio this one will be of interest you.

Being someone who is starting out using Visual Studio to develop parallel tasks, I’ve watched some demonstration of the support for parallel tasks in Visual Studio. That was fine, but when I came to get those nice windows up for my project I could not find where they were hidden.

The Parallel Task and Parallel Stack Windowsclip_image002

The only way I have found to get these windows displayed is as follows:

  1. Put some breakpoints into the code which has parallel activity. I put these break points into the “bits” of code I know are off on parallel tasks.
  2. Start debugging the program.
  3. Go to the Debug Tools Tool Bar, and select the “Breakpoints” icon. This then presents the dropdown menu with the parallel tool windows.
  4. There you can see there are keyboard short cuts to open the windows. This is fine if you can get the menu up and read the shortcuts, but I could never find the option to get the menu up. Why, these items only appear when the program is in “debug mode”.



  • This is another one of those, “easy when you know how” type of things. It is not entirely obvious when you start looking for the Parallel tools where to find them.
  • There are probably (undoubtable) other ways to get these windows up, but this is the one which I have found.

, , , , , , , ,

1 Comment

Multithreaded Programming in .Net with C# – Aggregated Exceptions


This may seem like an odd topic for inclusion in my multithreaded in C#, with the .Net 4.0 framework. If you read on you will see why this is  a topic for consideration when programming using the .Net 4.0 Framework. When you see why the designers of the .Net Framework included the AggregateException Class you will see that there is a use case here which needed to be addressed.


My previous blog post in this series was: Parallel Programming in .Net – Resources.

The Prompt For this Blog Post

Another resource:   Aggregating Exceptions or

This is an interesting article on the AggregateException Class. This class is heavily used in the .Net Framework parallel support classes. Why? Read on for some explanations as to why the .Net Framework needed aggregated exceptions.

Single Threaded Exception Model

In single the single threaded programming model, one thread can only throw one exception at a time.

The following is a very simple example of exception processing in a single threaded environment. What it does show is that only one exception can be “active” at once.

private void method_a()
    catch (NotImplementedException ex)

private void method_b()
    throw new NotImplementedException();

Multiple Threaded Exception Model

In the multiple threading model you can potentially have multiple exceptions being thrown (one from each thread executing).

The following is a very simple example. There are a couple of point to note:

  • I don’t know how many exceptions will be thrown.
  • The number of exceptions depends on a number of factors wedged into the Parallel.For method.
    • How many CPU’s (or hardware threads) the machine has. The implementation scales out to what the machine has.
    • The implementation could called from another Parallel.For (or any other Framework Parallel method – including Parallel LINQ), so again the number of threads available is unknown.
    • The implementation has it’s own scale across algorithms. So again depending on the way the algorithm reacts to the current workload, we could get any number of threads.
private void root_Method()
    Parallel.For(1, 20, a=> {
        throw new NotImplementedException(a.ToString());

There are a number of things which should be made clear. These are:

  1. We don’t know how many exceptions the caller should get.
  2. The ordinary exception cannot cater for more than one “meaning”. The Exception Class has an (just one) inner exception only.
  3. How does the implementation of Parallel.For decide which exception it should pass back from an arbitrary delegate (or, any code you like).

Why We Need Aggregated Exceptions?

The above starts to outline the use case for an aggregated exception, or more to the point an exception which can contain multiple exceptions. When you have multiple threads of execution each of those threads could throw an exception. These exceptions should go back to the thread which propagated the multiple threads. The caller should be “told” about all of the problems which occurred. As a result the .Net Framework “packs” all of these exceptions which occurred in the parallel operations into one exception which is passed (thrown) back to the caller. Acting as a container for all exceptions generated in a function is the function which the AggregateException Class performs.

The Structure of the AggregateException Class Class

The following code snippet is lifted from the MSDN article Aggregating Exceptions. I have included the code here so that I can highlight some of the additions that the AggregateException Class has, when compared to the “normal” exception classes.

[DebuggerDisplay("Count = {InnerExceptions.Count}")]
public class AggregateException : Exception
    public AggregateException();
    public AggregateException(params Exception[] innerExceptions);
    public AggregateException(IEnumerable<Exception> innerExceptions);
    public AggregateException(string message);
    public AggregateException(string message, Exception innerException);
    public AggregateException(string message,
        params Exception[] innerExceptions);
    public AggregateException(string message,
        IEnumerable<Exception> innerExceptions);

    public AggregateException Flatten();
    public void Handle(Func<Exception, bool> predicate);

    public ReadOnlyCollection<Exception> InnerExceptions { get; }

AggregateException.InnerExceptions Propertyimage

This is where the exceptions from the multiple threads are stored. There is a subtle nuance in this storage structure, which is that it is a hierarchical representation of the exceptions thrown.

The following diagram attempts to show that one AggregatedException can contain other Aggregated Exceptions, and ordinary exceptions. The .Net Framework will generate another “level” (insert another aggregated exception) when there is another level of the parallel support in the Framework is invoked. For example, a Parallel.For which invokes a child Parallel.For. These parent child relationships are preserved in the aggregated exception structure which the Framework generates.

AggregateException Constructor

It is worth noting that the constructors for the AggregatedException include the ability to construct an AggregatedException from collections of exceptions. As you would expect, when building a container object.

AggregateException.Flatten Method

This is an interesting method which has been included into the implementation of the AggregatedException. It simply flattens the hierarchal, or nested, structure which the AggregatedException contains into a single level representation. This representation is then ideal for applying LINQ to the inner exceptions collection, and “pulling” out the exceptions you are interested in.

The following is a non functional piece of code, but demonstrates the use of the Flatten method with LINQ.

AggregateException sample = new AggregateException();
var interesting_exceptions = from ex in sample.Flatten().InnerExceptions
                             where ex is DivideByZeroException
                             select ex;

AggregateException.Handle Method

The handle method is another way of working with the AggregatedException an the collection of exceptions which it can contain. This method produces an aggregated exception which contains just the exceptions which do not satisfy the predicate supplied.

    sample.Flatten().Handle(e => e is DivideByZeroException);
catch (AggregateException ex)
    // Sample minus the DivideByZeroExceptions


I hope that this discussion of the AggregateException Class proves useful.

It is a part of the .Net 4.0 Framework which undoubtable any developer producing multithreaded code will need to deal with.

, , , , , , , , ,

Leave a comment

Parallel Programming in .Net – Resources


This may be the first of a series of blog post on Parallel Programming with .Net 4.0. It is a “maybe” at this point in time, in that I intended to get into this topic over the coming weeks, but one never knows what else may crop up.

This started off being a look at the synchronised collections support in .Net 4.0, but I’ve decided that I should include some other topics. The other big inclusions are intended to be:

  • Parallel Looping constructs,
  • Parallel LINQ, which the parallel looping leads into, and
  • The Task Class addition to .Net 4.0 Framework.

When you include those, you pretty much have a Parallel programming .Net 4.0 “in a nutshell”. This series may have the impetus to look into some of the more “esoteric”, “subtle” and all-round deeper issues associated with the development, implementation, testing, and deployment of parallel solutions in .Net. But, wait and see, I could come up with some “gems”, and “pearls of wisdom” on the way.


This is the start of a list of “good” resources on parallel programming in .Net. I’ll try and post further blog posts on the topic as I find more resources (I’ve some PFD’s salted away on the topic as well I’ll try and dig up).

Parallel Programming with Microsoft .NET This is a MSDN Patterns and Practices publication. It contains very good information on parallel programming.

System.Collections.Concurrent Namespace This is the .Net namespace which contains some very useful “pre- cooked” collections which can be used in concurrent (parallel) programming. My preference is to always look at these collections first, and build my own concurrent collection as a last resort.

System.Threading.Tasks Namespace This is the >net namespace which contains the Task object, and the associated infrastructure built into .Net for running parallel Tasks. Again, my preference is to look here first, and only resort to using the Thread Class as a last resort when things do not fit with supplied .Net support.

Parallel Programming in the .NET Framework This is a “top level” page, which has links to much of the .Net Parallel support.

Threading in C# – Part 5 – Parallel Programming This looks to be very good on LINQ and Parallel aspects which can be invoked in the LINQ context. There is also some details on the Task, and associated .Net Framework classes. I’ll be reading this very carefully (again, it will not for the first time I refer to these pages).

Patterns Of Parallel Programming This is a link to a PFD which contains “much” detail on C# .Net Parallel Programming. I’ve yet to digest this one (118 pages is a bit much for one sitting, I’ll have a couple of “bites at this cherry”).

This is just a start of a list of resources. I’ll post further lists of resources, as I dig into the topics I wish to cover.

, , , , , , ,


%d bloggers like this: