I’ve been doing a lot of thinking about how Paint.NET needs to change or evolve before it hits version 4.0, and the big thing that keeps coming up is isolation. Right now, the control flow and error handling in Paint.NET lets one component crash or corrupt the whole program. This could be code sitting in a plugin that causes an unhandled exception in a way that my try/catch harnesses don’t handle, or it could be a COM component loaded by Windows common file Open/Save dialogs that crashes or locks up. Or maybe a piece of code doesn’t throw an exception but it does corrupt some other data, and then that corruption causes a crash later. And then that gets attributed to the wrong component. GDI+ seems to like doing that a lot.
Not only does Paint.NET need to be protected by crashing or "bad" plugins, but it needs to be protected from itself! Things like the Open/Save and Print dialogs are a good example, especially because they deal with external, "native" components. Some people have shell extensions installed that cause the Open/Save dialogs to fail, hang, or crash, and this breaks Paint.NET! This could be something as seemingly benign as a thumbnail provider for DivX videos. Yes, I get crash reports on the forum for that every so once in awhile. Why should Paint.NET have to crash if there’s a corrupted DivX video in a folder you’ve navigated to via File->Open? I’d rather crash just the Open dialog, detect the error without corruption in the main process, and then re-run the dialog in some kind of compatibility or reduced functionality mode. At least that way if it corrupts memory, then that corruption only affects the hosting process that it lives in.
So, there needs to be a really easy way for me to write a component in Paint.NET — whether it’s "built-in" or a "plug-in" — that can be isolated and not cause problems for the rest of the application. This means I want to host it in an external process and communicate with it, and then have a layer of abstraction in place for creating a "remote", isolated object.
The new System.AddIn namespace in .NET 3.5 provides a whole system for this, but honestly I find it to be way too complex, confusing, and involved. They have host views, add-in views, host-side adapter, add-in side adapters, etc. It’s a bunch of terminology that seems invented — why not just say client and server, proxy, tear-off, etc.? Every time you add a new interface or type you have to add code that spans at least 3 assemblies. Plus, they do not provide the ability for me to be running in a 64-bit process and have it communicate with an add-in that requires 32-bit hosting. This is important for certain plug-ins which are doing native interop to DLL’s that are only available in 32-bit (the VTF File Type Plugin comes to mind). They also do not provide the ability to activate an add-in that needs to run at a higher security level — for instance, if I want to require my Settings dialog to run with administrator privilege. I don’t want to have to deal with UAC at the code level, and would prefer it to be abstracted away. They also enforce a specific directory structure, and so on … anyway, it’s just too much and also too little.
.NET Remoting isn’t really an end-to-end solution for what I need, as it does not provide a hosting process. COM provides component hosting in an external process, but it seems like major overkill to register my .NET components as COM-proxied objects. Windows Communication Foundation is more for "enterprise" remoting and communication scenarios.
So it looks like I may have to roll my own solution, using System.IO.Pipes (actually I may prefer an HWND and WM_COPYDATA for marshaling) and borrowing heavily from the lessons put forth by System.AddIn (they have a very good story for versioning), and learned from my inbox full of crash logs. In any case it must be something that is easy to use by both the "host" and the "add-in" (or "built-in" :)). I’m thinking code generation will be very handy here in order to be able to declare an isolatable interface and then auto-generate the local tear-off/proxy, client stub, server stub, and registration glue.
paint.net blog 
I also find the System AddIn abit complicated. They do however let your add-in run in a different AppDomain.
There is also a AddIn Pipeline Builder project at http://www.codeplex.com/clraddins you might look at. You may be able to supply something similar for your codegen.
It’s particularly amazing how much work goes into managing events with the System.AddIn glue.
Not having looked at your other stated options, do you know or would you suspect that managing events might be easier using an alternative approach?
Thanks and keep up the good work.
I too have been dipping my toes recently into the world of System.AddIn. My gut reaction is to stick with it though (and as eilert mentions, tools exist already so you don’t have to write your own). Far too many times have I decided that the framework *almost* included what I wanted, so wrote my own implementation. It just is not worth all the effort of writing the code, finding all the bugs, and finding all the corner-cases, just to realize why the framework team wrote it like they did.
Andy — Although sometimes the learning experience of that can be very valuable. It depends on your project though — you often don’t have the flexibility to backtrack or make mistakes that affect something as pervasive as your remoting object model.
I have thought about this myself for a project I wanted to make. I finally decided I would have to make my own system. I have not started yet, but since then I have been considering writing it in C++ with something like DLLs or ELF shared object files instead of C# and the CLR. If you write this and make it open source, I might use the CLR and your system.
Its nice to see I’m not the only one turned off by System.AddIn. That said, it probably does handle its own particular scenario fairly well – the ability to have addins that execute in their own appdomains.
Definitely a v0/beta product, and it should *never* have been released as part of the .NET framework.
The only porob. is that whatever you do the appdomain isolation is not enough and you still will be able to crash the app. Sadly currently only seperation to a different process will allow you the ability to be safe from crash.
System.Addin sucks – it is way too complicated to achive an appdomain isolation and it sill does not guard you – try for yourself to through a system exception and you’ll crash the host.