Sunday, May 8, 2011

Introduction to IL Rewriting with Cecil, Part 1–Rewriting FizzBuzz and the Art of Redirecting Method Calls

lesson1

The simplest possible code example that anyone can learn from.

Introduction

When I first started learning IL rewriting and Cecil about several years ago, one of the difficulties that I struggled with was the fact that there were very few practical examples on how to take an existing assembly and modify it at runtime. In many ways, I was stranded in heavily undocumented territory, and needless to say, this lack of documentation made it very difficult to learn how to do anything useful with Cecil.

Meanwhile, in the Year 2011…

It’s now 2011, and I think it’s safe to say that for many people, IL rewriting (much less Cecil) is still a “big mystery wrapped in an enigma containing frustration”. Indeed, Cecil is an incredible library that can let you do some incredible things, but at the same time, it can be very frustrating since the learning curve is still steep and there are still no practical guides for using it. As a user, there must be some sample code out there that shows how to do the most basic tasks with Cecil, right?

Establishing the Feedback Loop

In order to learn any skill (such as IL rewriting), we need to establish a simple feedback loop that allows users to easily experiment with the tools they are given so that they know:

  • What went wrong if it doesn’t work
  • Where to fix it if it breaks
  • How to see the results of their experiments without getting mired in the implementation details of the tests themselves

In this case, we’ll need to set up a basic environment that will let users experiment and learn how to modify assemblies at runtime with Cecil. We will need:

  1. A test fixture that loads a sample assembly and gives users the chance to modify it before reloading the modified assembly into memory (An NUnit base fixture)
  2. A way to display/diagnose any invalid assembly errors that occur due to making changes to the original assembly (PEVerify)
  3. To make it easy to change so that we can experiment with different approaches to modifying IL, thus “closing” the feedback loop

Lost in Bytecode

Given these requirements, where would we even begin? It’s not every day that one decides to randomly parse .NET assemblies and learn how to change the underlying bytecode that ultimately defines their behavior. This can seem like a daunting task for even the most intelligent of budding interlopers, but fortunately for my readers, most of the work has already been done for you in these examples. All you need to do is sit back and scroll down the page, as I proceed to tell you the “ins” and “outs” about Cecil, and the practical lessons learned from rewriting IL. With that in mind, let’s get started!

A WriteLine for Another WriteLine

One of the simplest things that you can possibly do with Cecil is to swap a single static method call for another static method call with the same parameters and the same return type. (It’s a fairly simple operation since both methods have the same signature, and you don’t need to add any additional instructions to make it happen). In this case, I opted to swap all calls to Console.WriteLine() with calls to FakeConsole.WriteLine():

As you can see from the example above, I used a simple LINQ query to identify all the call instructions that needed to be modified. More experienced Cecil users will probably notice that I decided to rewrite all method calls to point to FakeConsole.WriteLine() instead of individually checking to make sure that the method call I was replacing was indeed Console.WriteLine(). Indeed, that was an intentional move, given that the FizzBuzz.Print() method doesn’t make any other external calls to any other methods besides Console.WriteLine().

Assuming that I somehow created an instruction that caused an invalid modified assembly, however, how would I be able to know what went wrong, much less know how to fix it?

PEVerify, how do I love and hate thee…

As it turns out, there is a tool called PEVerify.exe that can tell you whether or not the assemblies that you modify with Cecil are valid or invalid. For example, if I were to remove all the IL instructions out of the FizzBuzz.Print() method, PEVerify would give me the following error message:

peverify1

(Believe me, it’s much prettier when it’s zoomed out)

PEVerify will examine any given assembly and be able to tell you whether or not the compiler (or in this case, you, the human compiler) made any mistakes in creating the assembly. It can be a very useful tool, and that’s why I modified the sample test fixture to run PEVerify right after the user modifies the sample assembly. If you don’t already have PEVerify installed, make sure you download it and configure the Lesson1 app.config file to point to where PEVerify is installed:


Once PEVerify has been configured as part of the tests, the rest is up to your imagination.

Exploring Method Replacement and Beyond

Now that the basic IL rewriting setup has been laid out for you, the onus is on you to explore the possibilities with Cecil and IL rewriting, even if it means that you have to start with some small, basic steps. In the next installment in this series, I’ll show you how to use PEVerify and Cecil to keep the stack balanced so that you can do things like swap static method calls for instance method calls, and even do things like install runtime hooks so you can change your code as your application is running. Stay tuned!

9 comments:

  1. Philip, thank you for your posts on Mono.Cecil. They are invaluable.

    Most examples of method pointcuts that I see seem to intercept within the method rather than wrapping the call site. In this particular example, I have a call to a .NET framework class constructor that I want to preface with a call to a fixer.

    internal class Program
    {
    private static void Main(string[] args)
    {
    SerialPortFixer.Execute("COM1");
    using (SerialPort port = new SerialPort("COM1"))
    {
    port.Write("test");
    }
    }
    }

    (See http://zachsaw.blogspot.com/2010/07/serialport-ioexception-workaround-in-c.html for complete code.)

    Are you aware of any samples that demonstrate matching a call and then inserting new code before that call? The reason to do so in this case would be that the .NET assemblies are signed and cannot be resigned. Ultimately, doing this could be implemented as a post compile MsBuild task.

    Thanks
    Rich

    ReplyDelete
  2. I would love to see more Cecil examples, keep them coming man, there is so little out there on it, when its so powerful

    ReplyDelete
  3. Philip,

    I saw on Twitter that you started using BDD for your specifications/acceptance tests. Can you give a few examples of specs? There will be a TDD vs. BDD session in Oslo tomorrow, and I am collecting examples. If you don't mind I can show your example of using BDD (with reference to you of course).

    Vagif Abilov

    ReplyDelete
  4. Hi Vagif,

    I haven't started using specs over traditional unit tests, but you're more than happy to show Tao (that is, the TDD version of Tao) as an example of how traditional TDD can get you mired in testing individual units of functionality while losing sight of the bigger picture. One of the reasons why I opted to rewrite Tao using BDD specs is that I needed to focus more on how the system should behave, and I also wanted to use specifications to set the boundaries of where the development work should end.

    For me, the switch to BDD makes more sense since I need to define the scope of the Tao project from the outset so that I don't keep extending it indefinitely. TDD can be just as effective, but IMO, shifting from the micro-design (TDD) to a macro design perspective can have a dramatic impact on how developers work, and that's why I'm sticking with BDD.

    ReplyDelete
  5. Hi Philip,

    Thanks for the quick reply. If you don't mind I will refer to Tao in my presentation with a quote from you about decision to rewrite it using BDD specs.

    ReplyDelete
  6. Sure, you can go ahead and quote me, Vagif

    ReplyDelete
  7. I have been searching for a while for modifying .net code in runtime. I currently use .net Profiling API to re-write IL at runtime. Can this be done using Cecil?
    Some thing like this
    static void FunctionToBeModified(object arg1, object arg2)
    {
    BeginInjectionMethod(new object[] {arg1, arg2});
    ActualMethodImplementation(...)
    EndInjectionMethod();
    }

    where BeginInjectionMethod and EndInjectionMethod are the methods to be instrumeted at runtime.
    Can this be done using Cecil?

    ReplyDelete
  8. Hi Jasper,

    You can do that kind of instrumentation at runtime if you inject the hooks into the assembly in memory with before loading it. That's how LinFu.AOP adds before/after/around method behaviors at runtime. The trick is to make sure that the hooks that you inject are all interface calls so that you can plug in your custom behaviors at runtime. HTH

    ReplyDelete
  9. Thanks Philip. Your LinFu.AOP is great.
    Currently I use .Net profiling API. Writing IL is not my forte really. But with LinFu it will make my life a lot more simpler I think.

    How can I instrument .net framework classes? Also how to use linfu in such cases where I need to instrument .net framework classes and for classes which don't have source code. All in runtime. Are there any examples which u made available?

    Thanks in advance.

    -Jasper

    ReplyDelete

Ratings by outbrain