.NET

Azure Queues: Retry Poison Messages

by

Currently, the Microsoft Azure Storage Explorer doesn’t have the ability to move messages from one Azure queue to another. During development, I sometimes need to retry messages that are in the poison message queue. A poison message is a message that has exceeded the maximum number of delivery attempts to the application. This happens when a queue-based application cannot process a message because of errors.

It’s not difficult to write, but thought it might save someone a few minutes, so I’m posting it here. You’ll need to add a NuGet package reference to Microsoft.NET.Sdk.Functions


using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Queue;

void Main()
{
    const string queuename = "MyQueueName";

    string storageAccountString = "xxxxxx";

    RetryPoisonMesssages(storageAccountString, queuename);
}

private static int RetryPoisonMesssages(string storageAccountString, string queuename)
{
    CloudQueue targetqueue = GetCloudQueueRef(storageAccountString, queuename);
    CloudQueue poisonqueue = GetCloudQueueRef(storageAccountString, queuename + "-poison");

    int count = 0;
    while (true)
    {
        var msg = poisonqueue.GetMessage();
        if (msg == null)
            break;

        poisonqueue.DeleteMessage(msg);
        targetqueue.AddMessage(msg);
        count++;
    }

    return count;
}

private static CloudQueue GetCloudQueueRef(string storageAccountString, string queuename)
{
    CloudStorageAccount storageAccount = CloudStorageAccount.Parse(storageAccountString);
    CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
    CloudQueue queue = queueClient.GetQueueReference(queuename);

    return queue;
}

Does the Microsoft CSP RSA provider support a maximum of 2048 bits?

by

It seems that the maximum Microsoft’s RSA provider supports is 2048 bits: Is there a limit for RSA key exponents in .NET?

I’ve just been been experimenting with 3072 and 4096 bit keys without success. Error message is “Bad Data” !

Nice one Microsoft! Thanks for you concern about security!

Surprising since NIST’s recommendations are that 3072 bit keys are required “if security is required beyond 2030”

C#: Simple Read/Write Application Settings

by

I needed a simple C# class for saving writable application settings. The first attempt was to use Project -> Properties -> Settings, but only user scoped settings are writable. Here’s a simple solution: you will need to add Newtonsoft’s Json.NET (Nuget Json.Net) to your project, if you are not already using it.

using System.IO;
using Newtonsoft.Json;
...

public class AppSettingsBase<T> where T : new()
{
    private const string DefaultSettingsFilename = "settings.json";

    public void Save(string fileName = DefaultSettingsFilename)
    {
        File.WriteAllText(fileName, JsonConvert.SerializeObject(this));
    }

    public static void Save(T settings, string fileName = DefaultSettingsFilename)
    {
        File.WriteAllText(fileName, JsonConvert.SerializeObject(settings));
    }

    public static T Load(string fileName = DefaultSettingsFilename)
    {
        return (File.Exists(fileName)) 
            ? JsonConvert.DeserializeObject<T>(File.ReadAllText(fileName)) : new T();
    }
}

Add your application defined settings to your settings class (derived from the base template):

public class AppSettings : AppSettingsBase<AppSettings>
{
    public Guid ClientId = Guid.Empty;
    public List<string> ServerList = new List<string>();
    public List<string> DatabasesToExclude = new List<string>();

    public bool IsAutoUpdate = true;
    public int AutoUpdaterCheckIntervalHours = 6; 
    public DateTime LastUpdateCheckTime = DateTime.MinValue;

    ...
}

Using your settings is simple:

    AppSettings settings = AppSettings.Load();

    settings.LastUpdateCheckTime = DateTime.UtcNow;
    settings.Save();

Create a Single .NET Executable with Multiple Assemblies

by

If you want to create a single .NET executable there are several possibilities. In the past I’ve used Jeffrey Richter’s technique of embedding DLLs as resources and loading these into memory from the Manifest Resource Stream on startup. While this works well, it does require a bit of extra code to not try to access anything in an assembly until after it’s been loaded at startup.

In the interest of cleaner code, I decided to give ILMerge a go.

It’s simple to install ILMerge into your application using NuGet:

PM> Install-Package ilmerge -Version 2.14.1208

To run ILMerge on your release builds, add an AfterBuild section to your default projects .csproj file:

  <Target Name="AfterBuild" Condition=" '(Configuration)' == 'Release' ">     <CreateItem Include="@(ReferenceCopyLocalPaths)" Condition="'%(Extension)'=='.dll'">       <Output ItemName="AssembliesToMerge" TaskParameter="Include" />     </CreateItem>     <PropertyGroup>       <ReferenceAssemblies>C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.2</ReferenceAssemblies>     </PropertyGroup>     <Message Importance="high" Text="Executing ILMerge...with target platform from(ReferenceAssemblies)" />
    <Exec Command="&quot;(SolutionDir)\packages\ILMerge.2.14.1208\tools\ILMerge.exe" /target:exe /out:@(MainAssembly) /internalize /targetplatform:v4,"(ReferenceAssemblies)&quot; &quot;@(IntermediateAssembly)&quot; @(AssembliesToMerge->'&quot;%(FullPath)&quot;', ' ')" />
    <Delete Files="@(ReferenceCopyLocalPaths->'$(OutDir)%(DestinationSubDirectory)%(Filename)%(Extension)')" />
  </Target>

Note: If you receive this error from ILMerge when building your release executable:

An exception occurred during merging:
ILMerge.Merge: There were errors reported in AWSSDK.Core’s metadata.
Array dimensions exceeded supported range.
at ILMerging.ILMerge.Merge()
at ILMerging.ILMerge.Main(String[] args)

It’s because ILMerge doesn’t currently support the new portable PDB format.
https://github.com/Microsoft/ILMerge/issues/29
https://github.com/Microsoft/ILMerge/issues/11

If you can live without the .pdb files being merged into a single PDB file add /ndebug to the Exec command.

This is a temporary workaround for issues with ILMerge where it cannot process the new portable .NET PDB format.

An excellent resource on PDB files is John Robbins’s : PDB Files: What Every Developer Must Know :

A .NET PDB only contains two pieces of information, the source file names and their lines and the local variable names. All the other information is already in the .NET metadata so there is no need to duplicate the same information in a PDB file.

How to send large amounts of data securely

by

RSA public/private key cryptography is limited in the amount of data that it can encrypt. With the commonly used v1.5 padding and the largest RSA key (currently 2048-bits), the maximum size of data that can be encrypted is 245 bytes.

If you want to encrypt and securely send more data than that you need to use a combination of asymmetric and symmetric encryption algorithms.

In practice, RSA is used to exchange an encrypted key between communicating endpoints that is then used to symmetrically encrypt/decrypt the large data.

Encryption by sender:

  1. Generate a cryptographically strong random key, K, of the length required for a symmetrical encryption technique such as Rijndael (maximum key size is 256 bits for Rijndael).
  2. Symmetrically encrypt your data using Rijndael using the random key generated in step 1.
  3. Using RSA, asymmetrically encrypt the random key generated in step 1 with the public part of the recipient’s RSA key.
  4. Send the RSA encrypted random key AND the encrypted data from steps 2 and 3 to recipient.

Decryption by recipient:

  1. Decrypt the encrypted key using your private RSA key.
  2. Decrypt the original data using the RSA-decrypted symmetric key from step 1.

Here’s how to generate a 2048 bit public/private key in C#:

    using (rsaProvider = new RSACryptoServiceProvider(2048))
    {
        rsaProvider.PersistKeyInCsp = false;

        // Export public key to file
        publicKey = rsaProvider.ToXmlString(false);
        using (publicKeyFile = File.CreateText(publicKeyFileName))
        {
            publicKeyFile.Write(publicKey);
        }

        // Export private/public key pair to file
        privateKey = rsaProvider.ToXmlString(true);
        using (privateKeyFile = File.CreateText(privateKeyFileName))
        {
            privateKeyFile.Write(privateKey);
        }
    }

log4net: .NET Logging Tool

by

I took a brief look at log4net quite some time ago and have to admit I didn’t look closely enough! log4net is a port of the highly successful java based log4j logging library and is very mature tool. It’s been at the back of my mind to have another look, and my current project provided the opportunity and impetus. Without wanting this to sound like hype, log4net really does provide a one-stop shop for all your logging, tracing and diagnostics in a simple to use package.

It’s very easy to use out of the box, and highly intuitive. It comes with just about every ‘appender’ you could want (an ‘appender’ is a kind of output sink, i.e. a target for your logging messages). You can define the layout of the log messages. It’s extensible, if you require some other type of appender. It comes with the source code. It has excellent documentation. It’s free! If that doesn’t sound like a great deal, well, there’s no satisfying some people!

I’m going to be using and recommending log4net for all new .NET projects I work on.

You can download log4net here. Even though log4net is fully extensible, it is highly likely that it will meet your logging needs straight out of the box.

The excellent documentation contains config-examples for the list of currently supported appenders:

Of these, probably the most commonly used are the:

The documentation and tutorials section provide examples of how to configure the various appenders (\examples\net\1.0\Tutorials\ConsoleApp\cs\src, for example).

[Note: OutputDebugString is also worth looking at; it’s a Win32 API call that’s been there since before .NET, and is a nice technique for connecting to live applications and capturing logging on the fly using a separate application to catch debugging messages]

Adding log4net to your VS2005 project

Download log4net from the official site here. Unzip it to create a log4net folder (with the version number appended) containing the binaries, documentation, examples and source code.

Add the log4net assembly to your project

It’s good practice to include third party assemblies under source control as part of a project. That way, performing a get latest on a new PC includes everything.

If your project does not already contain a source-controlled external libs folder for third party assemblies:

  1. Right click on project
  2. Select “Add -> New Folder”, call it “ThirdPartyDlls” or “libs” or whatever your in-house standards specify.

OK, now add log4net:

  1. Right click on your external libs folder, select “Add Existing Item…” and browse to where you unzipped log4net and choose the release version of log4net.dll (\bin\net\2.0\release\log4net.dll)
  2. Right click on References, select “Add Reference …” and browse to your libs folder and pick the log4net.dll you just added.

Configure log4net: Creating and Specifying the Log4Net Config File

Although it is possible to add your log4net configuration settings to your project’s app.config or web.config file, it is preferable to place them in a separate configuration file. Aside from the obvious benefit of maintainability, it has the added benefit that log4net can place a FileSystemWatcher object on your config file to monitor when it changes and update its settings dynamically.

To use a separate config file, add a file named Log4Net.config to your project and add the following attribute to your AssemblyInfo.cs file:

[assembly: log4net.Config.XmlConfigurator(ConfigFile = “Log4Net.config”, Watch = true)]

Note: for web applications, this assumes Log4Net.config resides in the web root. Ensure the log4net.config file is marked as “Copy To Output” -> “Copy Always” in Properties.

Here is an sample log4net config file with several appenders defined:

<?xml version="1.0" encoding="utf-8" ?>
 
<log4net>
  <appender name="RollingFileAppender" type="log4net.Appender.RollingFileAppender">
    <file value="..\\Logs\\CurrentLog" />
    <appendToFile value="true" />
 
    <rollingStyle value="Size" />
    <maxSizeRollBackups value="10" />
    <maximumFileSize value="10000" />
    <staticLogFileName value="true" />
    <!-- Alternatively, roll on date -->
    <!--  -->
    <!--  -->
    <filter type="log4net.Filter.LevelRangeFilter">
      <acceptOnMatch value="true" />
      <levelMin value="INFO" />
      <levelMax value="FATAL" />
    </filter>
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%-5level %date [%thread] %-22.22c{1} - %m%n" />
    </layout>
  </appender>
 
  <appender name="LogFileAppender" type="log4net.Appender.FileAppender">
    <file value="log-file.txt" />
    <!-- Example using environment variables in params -->
    <!--  -->
    <appendToFile value="true" />
    <layout type="log4net.Layout.PatternLayout">
      <header value="[Your Header text here]" />
      <footer value="[Your Footer text here]" />
      <conversionPattern value="%date [%thread] %-5level %logger [%ndc] 
                 <%property{auth}> - %message%newline" />
    </layout>
    <!-- Alternate layout using XML            
     -->
  </appender>
 
  <appender name="ConsoleAppender" type="log4net.Appender.ConsoleAppender">
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger - %message%newline" />
    </layout>
  </appender>
 
  <appender name="EventLogAppender" type="log4net.Appender.EventLogAppender" >
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger
                         [%property{NDC}] - %message%newline" />
    </layout>
  </appender>
 
  <!-- Set the default logging level and add the active appenders -->
  <root>
    <level value="DEBUG" />
    <appender-ref ref="LogFileAppender" />
    <appender-ref ref="ConsoleAppender" />
    <appender-ref ref="RollingFileAppender" />
  </root>
 
  <!-- Specify the level for specific categories (“namespace.class”)-->
  <logger name="ConsoleApp.LoggingExample">
    <level value="ERROR" />
    <appender-ref ref="EventLogAppender" />
  </logger>
 
</log4net>

In the “RollingFileAppender“ defined above, as Phil Haack points out: “Note that the file value (with backslashes escaped) points to ..\Logs\CurrentLog. [In Web Applications] this specifies that log4net will log to a file in a directory named Logs parallel to the webroot. You need to give the ASPNET user write permission to this directory, which is why it is generally a good idea to leave it out of the webroot. Not to mention the potential for an IIS misconfiguration that allows the average Joe to snoop through your logs.”

Using the Logger

At the start of each class declare a logger instance as follows:

public class ClassWithLoggingExample
{
    private static readonly ILog log = 
LogManager.GetLogger( System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
 
    ...
}

You will need to add a “using log4net;” statement to each class file. By defining a logger in each class you have the ability to control the logging level on a class by class basis, simply by using the config file.

Once you have instantiated a logger, you can call its logging methods. Each method is named for the logging level. For example:

// In ascending order of severity (descending level of verbosity) 
log.Debug("This is a DEBUG level message. The most VERBOSE level.");
log.Info("Extended information, with higher importance than the Debug call");
log.Warn("An unexpected but recoverable situation occurred");
log.Error("An unexpected error occurred, an exception was thrown, or is about to be thrown", ex);
log.Fatal("Meltdown!", ex);

Whether or not a message shows up in your logs depends on how you have configured your appenders and the logging level you have set. If you want to avoid the overhead of string construction and a call to a logging method, you can first test to see if the appropriate level is active:

// If you are concerned about performance, test the appropriate
// log level enabled property (one for each of the log level methods)
if (log.IsInfoEnabled) 
    log.Info("Performance sensitive Info message");

The format of the logged output can be adjusted in the config file. See the log4net documentation that came in the zipped download (\doc\release\sdk\log4net.Layout.PatternLayout.html).

For web applications, add the following line to the Application_Error method in the Global.asax.cs file:

protected void Application_Error(object sender, EventArgs e)

{
    log.Fatal("An uncaught exception occurred", this.Server.GetLastError());
}

Contextual Logging

Log4net also has a mechanism called NDC (which stands for Nested Dynamic Context) where you can add contextual information to the log, allowing you to mark log entries as coming from a certain user or having a specific context. These can be nested in a push/pop stack fashion. For example:

// Push a message on to the Nested Diagnostic Context stack
using(log4net.NDC.Push(this.User.Name))
{
    log.Debug("log entry will be tagged with the current user's name");
}
// The NDC message is popped off the stack at the end of the using {} block

The context is added to log messages where the pattern layout contains “[%ndc]”.

Resources / References

If you run into permissions problems configuring logging for ASP.NET, check out Phil Haack’s post on configuring log4net for Web Applications here. His quick and dirty guide is also worth reading.

There is also a very useful log4net dashboard / viewer tool called l4ndash.

Bill Ryan has a blog post about The connection string for ADONetAppenders as well as his original post on a log4net walkthrough.

.Net C# Singleton Pattern – Best Practice

by

What is the best practice way to implement the Singleton design pattern in C#? You might be surprised at the number of flawed examples there are out there; many of the articles I looked at were incorrect (some subtlely, some not so) and several were on high traffic, popular sites!

This excellent article by Jon Skeet not only discusses several ways not to do it, but also the correct way:

public sealed class Singleton

{

// Prevent compiler adding a default public parameterless constructor

private Singleton() {}

public static readonly Singleton Instance = new Singleton();

// Explicit static constructor instructs compiler

// NOT to mark type as ‘beforefieldinit’

// Remove this if you do not need to guarantee lazy instantiation

static Singleton() {}

}

The static constructor is only required if you want to guarantee lazy instantiation (which Jon explains here.)

See also this MSDN article from Feb 2002.