A while ago, I needed to analyze a bunch of files based on the unique trimmed lines in them.

I based my code on the C# Tee filter and the StackOverflow example of C# deduping based on split.

It is a bit more extensive than strictly needed, as it has a few more commandline arguments that come in handy when processing files on the console:

DeDupe - Dedupes a file into unique lines (only the first occurance of a line is kept) standard output
Lines are terminated by CRLF sequences
C# implementation januari 5th, 2012 by Jeroen Wiert Pluimers (http://wiert.wordpress.com),

DeDupe [-i | --ignore] [-t | --trim] [-f | --flush] [-l | --literal] [-? | --h | --help | /?] [file0] [...]
   Example:
 DeDupe --trim file0.txt file1.txt
   Dedupes the appended content of file0.txt and file1.txt into standard output

-t | --trim                  Will trim the lines before considering duplicates
-f | --flush                 Flushes files every CRLF
                               (setting is per tee instance)
-i | --ignore                Ignore cancel Ctrl+C keypress: see UnixUtils tee
-l | --literal               Stop recognizing flags, force all following filenames literally
-? | --h  | /? | --help      Displays this message and immediately quits

Duplicate filenames are quietly ignored.
If no input filenames are specified, then standard input is used
Press Ctrl+Z (End of File character) then Enter to abort.

Here is the source code:

using System;
using System.IO;
using System.Collections.Generic;

namespace DeDupe
{
    class Program
    {
        static void help()
        {
            Console.Error.WriteLine("DeDupe - Dedupes a file into unique lines (only the first occurance of a line is kept) standard output");
            Console.Error.WriteLine("Lines are terminated by CRLF sequences");
            Console.Error.WriteLine("C# implementation januari 5th, 2012 by Jeroen Wiert Pluimers (http://wiert.wordpress.com),");
            Console.Error.WriteLine("");
            Console.Error.WriteLine("DeDupe [-i | --ignore] [-t | --trim] [-f | --flush] [-l | --literal] [-? | --h | --help | /?] [file0] [...]");
            Console.Error.WriteLine("   Example:");
            Console.Error.WriteLine(" DeDupe --trim file0.txt file1.txt");
            Console.Error.WriteLine("   Dedupes the appended content of file0.txt and file1.txt into standard output");
            Console.Error.WriteLine("");
            Console.Error.WriteLine("-t | --trim                  Will trim the lines before considering duplicates");
            Console.Error.WriteLine("-f | --flush                 Flushes files every CRLF");
            Console.Error.WriteLine("                               (setting is per tee instance)");
            Console.Error.WriteLine("-i | --ignore                Ignore cancel Ctrl+C keypress: see UnixUtils tee");
            Console.Error.WriteLine("-l | --literal               Stop recognizing flags, force all following filenames literally");
            Console.Error.WriteLine("-? | --h  | /? | --help      Displays this message and immediately quits");
            Console.Error.WriteLine("");
            Console.Error.WriteLine("Duplicate filenames are quietly ignored.");
            Console.Error.WriteLine("If no input filenames are specified, then standard input is used");
            Console.Error.WriteLine("Press Ctrl+Z (End of File character) then Enter to abort.");
        }

        static void OnCancelKeyPressed(Object sender, ConsoleCancelEventArgs args)
        {
            // Set the Cancel property to true to prevent the process from
            // terminating.
            args.Cancel = true;
        }

        static List<String> filenames = new List<String>();

        static void addFilename(string value)
        {
            if (-1 == filenames.IndexOf(value))
                filenames.Add(value);
        }

        static bool trimLines = false;
        static bool flushFiles = false;
        static bool stopInterpretingFlags = false;
        static bool ignoreCtrlC = false;

        static int Main(string[] args)
        {
            try
            {

                foreach (string arg in args)
                {
                    //Since we're already parsing.... might as well check for flags:
                    if (stopInterpretingFlags)  //Stop interpreting flags, assume is filename
                    {
                        addFilename(arg);
                    }
                    else if (arg.Equals("/?") || arg.Equals("-?") || arg.Equals("-h") || arg.Equals("--help"))
                    {
                        help();
                        return 1; //Quit immediately
                    }
                    else if (arg.Equals("-t") || arg.Equals("--trim"))
                    {
                        trimLines = true;
                    }
                    else if (arg.Equals("-f") || arg.Equals("--flush"))
                    {
                        flushFiles = true;
                    }
                    else if (arg.Equals("-i") || arg.Equals("--ignore"))
                    {
                        ignoreCtrlC = true;
                    }
                    else if (arg.Equals("-l") || arg.Equals("--literal"))
                    {
                        stopInterpretingFlags = true;
                    }
                    else
                    {	//If it isn't any of the above, it's a filename
                        addFilename(arg);
                    }
                    //Add more flags as necessary, just remember to SKIP adding them to the file processing stream!
                }

                if (ignoreCtrlC) //Implement the Ctrl+C fix selectively (mirror UnixUtils tee behavior)
                    Console.CancelKeyPress += OnCancelKeyPressed;

                HashSet<string> keys = new HashSet<string>();
                Int64 index = 0;

                using (StreamWriter writer = new StreamWriter(Console.OpenStandardOutput()))
                {
                    if (filenames.Count == 0)
                        using (StreamReader reader = new StreamReader(Console.OpenStandardInput()))
                        {
                            processInputFileReader(keys, writer, reader, ref index);
                        }
                    else
                        foreach (String filename in filenames)
                        {
                            using (StreamReader reader = new StreamReader(filename))
                            {
                                processInputFileReader(keys, writer, reader, ref index);
                            }
                        }
                    writer.Flush();
                }

            }
            catch (Exception ex)
            {
                Console.Error.WriteLine(String.Concat("DeDupe: ", ex.Message));  // Send error messages to stderr
            }

            return 0;
        }

        private static void processInputFileReader(HashSet<string> keys, StreamWriter writer, StreamReader reader, ref Int64 index)
        {
            string line = readLine(reader);
            while (!string.IsNullOrEmpty(line))
            {
                string candidate = line;
                if (keys.Add(candidate))
                {
                    writer.WriteLine(line);
                    index += line.Length + Environment.NewLine.Length;
                    if (flushFiles)
                        writer.Flush();
                }

                line = readLine(reader);
            }
        }

        private static string readLine(StreamReader reader)
        {
            string line = reader.ReadLine();
            if (null != line)
                if (trimLines)
                    line = line.Trim();
            return line;
        }
    }
}

–jeroen

via: C# text file deduping based on split – Stack Overflow.

A while ago, I needed some generic way of parsing data that could result instancef of both regular ordinal and nullable ordinal types.

Luckily there was a nice question about this on StackOverflow, of which I liked this answer (source code below) much more than the accepted answer: concise, elegant, period.

    public static T? Parse(this string text) where T: struct
    {
        object o = null;
        try {
            var ttype = typeof(T);
            if (ttype.IsEnum)
            {
                T n = default(T);
                if (Enum.TryParse(text, true, out n))
                    return n;
            }
            else
            o = Convert.ChangeType(text, ttype);
        }
        catch { }

        if (o == null)
            return new Nullable();

        return new Nullable((T)o);
    }

–jeroen

When you get a C# Warning CS0067, the MSDN documentation tell you to insert a “#pragma warning disable 0067″, but Trevor Robinson has a better solution at C# Warning CS0067: The event ‘event’ is never used – Trevor’s Dev Blog – Site Home – MSDN Blogs:

Since the event is never used, make the event explicit, or even throw an exception in the add clause of the event.

–jeroen

Boy, I love stackoverflow.

Unlike a forum – which is focussed on discussion – stackoverflow is focussed on questions and answers, and they have lots of good answers.

Recently, I had to format numbers in a human readable way.

The formatting – C#: File-size format provider – Stack Overflow question had at least 3 good ways.

I chose the FileSizeFormatProvider answer, as it best fitted my goal and it is easy to extend, but the StrFormatByteSize answer works very well too if you need an ‘out of the box’ solution that works on native Windows.

–jeroen

When maintaining software, you can’t always use the latest versions of your tools and languages.

Since C# has added a lot since version 1, it is handy to know what changed in which version.

Of course nothing beets the (paper and electronic!) editions of C# in Depth (covering C# 1, 2 and 3) and C# in Depth second edition (covering C# 2, 3 and 4).

Those books are thick (they need to, there is a truckload to cover!).
(The publisher (Manning) actually did a very good job on the e-book versions. Highly recommended, as it makes searching stuff so much easier).

The answers to the c# – List of new features in C#2.0, 3.0 and 4.0 Stack Overflow question contains a few concise lists of major changes in the language.

I really like the Bluffer’s Guide to C# 2, and the Bluffer’s Guide to C# 3 that Jon Skeet mentioned in his answer (and wish he wrote a Bluffer’s Guide to C# 4 as well), as they focus on some essential stuff in a very concise way.

If you want more detailed info, start with these stackoverflow questions:

• C# 3
• C# 4

Since stackoverflow wasn’t there when C# 2 was introduced in 2005, start with this nice C# 2 overview article on CSharpCorner.

Note: contrary to popular belief, there is no C# 3.5. There is .NET 3.5 (adding LINQ) and .NET 3.0 (adding WPF, WCF and WF) all based on the C# 3.0 and CLR 2 SP1 features.
(Yes, some expression tree features don’t work on CLR2 RTM, as that one has a few bugs, see Jared’s comment on this answer).

–jeroen

via: c# – List of new features in C#2.0, 3.0 and 4.0 – Stack Overflow.

While browsing for some less used but useful C# features, I came across a great post by Michael (Mike) Ward from Blue Onion Software on Favorite C# Idioms based on the great Hidden Features of C# thread on StackOverflow.

Mike starts with

There’s a great post on Stackoverflow about hidden features in C#. I prefer to think of these as idioms rather than hidden features since most are actually documented. Here are my favorites.

Then he continues with some nice examples of C# idioms.

Recommended reading!

–jeroen

via Favorite C# Idioms | Blue Onion Software.