CSV Parser
I’m currently working on a project where I needed a C# console application that was able to read through a Excel CSV (Comma Separated Values) file.
Basically the CSV file format is just a txt file with rows and each column is then separated by a comma (surprise!) or a semicolon. Besides a comma the data in each column can optionally be “framed” by quotation marks.
Therefore i started out with the following code, just as I would read through a normal txt file:
| try{ using (StreamReader readFile = new StreamReader(path)) { // Do something here… } } catch (Exception e) { // Do some error handling here… } |
This is, as you can see, really straight forward. First of all I declare an object of a StreamReader in a using statement. Using the object “readFile” I am able then to navigate the file. The using statement is important as this will do the cleanup for me, by calling StreamReader.Dispose(), when the statement finishes. I always wrap this kind of code in a try…catch because when you work with files, errors just occasionally happen.
Now, to read the data from the CSV file I add the following lines of code inside the using statement:
| List<string[]> parsedData = new List<string[]>(); string line; string[] row; while ((line = readFile.ReadLine()) != null) parsedData.Add(row); } |
It just declares a new List that can hold an array of strings and the line and row variables is needed when traversing through the file. I then use the readFile object to call the ReadLine() method of the StreamReader class in a while loop. When there is no more lines in the file the line variable will be null. Inside the while loop I use the string.Split() method to split the line into an array of strings (my columns) and I then add this array to my List object (parsedData).
The problem then was that I didn’t know exactly what encoding the file would be in. What to do then? I settled on a solution where I tell the StreamReader what encoding the file probably has and it will then open it in that encoding. This can be done by adding a parameter when calling the constructor on the StreamReader class like this:
| using (StreamReader readFile = new StreamReader(path, encoding)) |
Finally all this can be wrapped in a nice method. I also added a check to be sure that the file I want to parse is actually available. But there you go:
| public static List<string[]> ParseCSV(string path, Encoding encoding, char splitter) { if (!File.Exists(path)) return null; List<string[]> parsedData = new List<string[]>(); try { { { return parsedData; } |
Alle brugere tæller
Der dumpede her til morgen et link ind i min mailboks til et blogindlæg af Dorte Toft, som jeg synes er meget interessant. Det handler om, at hun har et nytårsønske hvor IT skal være for alle inkl. dem der ikke er superbrugere eller har overskudet til at sætte sig ind i et stort nyt IT-system.
Blogindlæget kan læses her: http://www.business.dk/tech-mobil/nytaarsoenske-it-tumper
I bund og grund handler indlæget vel egentlig om det vigtige i at have alle brugere af et IT-system med på vognen fra første spadestik og måske endnu før dette tages i forbindelse med IT-projekter. Jeg kunne ikke være mere enig. Som Dorte Toft også skriver er der en række eksempler på, både i det private og det offentlige, at der ikke er blevet taget højde for den diversitet der kan være i de brugere der skal kobles på et system senere hen.
Jeg er dog ikke enig med Dorte Toft om, at man ikke skal regne med at finde forståelse for dette hos konsulenterne. Det må vel være i alles interesse på et projekt, at det bliver en succes hele vejen rundt? Desuden mener jeg det er vigtigt for en konsulent på et projekt ikke kun at pleje egne interesser, men selvfølgelig først og fremmest at give kunden det som kunden egentlig har brug for, både når det kommer til teknologi og brugervenlighed og interaktionen mellem disse.
Jeg fik et skrækeksempel, på noget af alt dette her i efteråret i forbindelse med mit studie. Her havde vi på et kursus en gæsteforelæser fra Dansk IT ude for at forklare hvordan de mener den offentlige sektor i Danmark skal digitaliseres. Her drejede samtalen sig hurtigt over på en af de store problemstillinger der opstår når dette igen og igen skal realiseres, nemlig de mange forskellige forudsætninger brugerne af en offentlig IT-service har. Her mente gæsteforelæseren fra Dansk IT, at man skulle fra det offentliges side i digitaliseringen gå efter, at få “en mand på månen” og så måtte man jo leve med “at alle ikke blev lige så kloge som Niels Bohr”, underforstået, at han mente man skulle gå efter at tænke så stort som muligt og så måtte man bare erkende, at alle danskere ikke ville kunne bruge de digitale IT-services. Da jeg i forbindelse med forelæsningen efterfølgende påpegede, at det umuligt kunne være i Danmarks interesse, at sætte en meget stor del af den danske befolkning af med denne tankegang, fik jeg svaret at det måtte man jo leve med hvis man ønskede at tænke så stort som muligt. Desuden var der fra gæsteforelæserens side en generel idé om at tingene skulle gøres nu og det skulle gøres hurtigst muligt. Det virker gang på gang, når der skal laves store IT-projekter i det offentlige, som om det er vigtigere at gøre tingene hurtigt i stedet for en gang for alle at gøre det ordentligt og bruge den tid på det som er nødvendigt for at produktet bliver i orden. For mig at se er den digitale tinglysning et skrækeksempel på netop det sidste.
Se i øvrigt den her artikel og bliv overrasket: http://www.business.dk/tech-mobil/danmark-er-klar-til-faa-mand-paa-maanen
Læg i øvrigt i ovenstående artikel mærke til sammenligningen mellem Carlsberg og McAffee som deres “gode eksempel”. Den er rent ud sagt tåbelig. Man kan overhovedet ikke sammenligne en produktionsvirksomhed og en IT-virksomhed på den måde. Det svarer til at sammenligne pærer og bananer. De er begge frugter, men derefter holder lighederne også op.
Meget hen ad vejen er det, for mig at se, en god forklaring på hvorfor så mange digitaliseringsprojekter i det offentlige herhjemme ikke bliver den succes som de var tiltænkt. Det er, som Dorte Toft skriver i sit blogindlæg, vigtigt at have alle med. Alle kan vel forhåbentlig se det logiske i, at det ikke nytter at afskære en stor del af den danske befolkning fra at benytte de offentlige services fx. pga. deres IT-kundskaber? For at digitaliseringen af det offentlige skal blive en succes skal man netop gøre det muligt for alle at benytte de digitale services der stilles til rådighed. Men det gælder jo også, som Dorte Toft er inde på, for dem der skal servicere borgerne både i administrationen i kommunerne og personalet på sygehusene, at de services de skal bruge en stor del af deres hverdag på, ikke i høj grad er tiltænkt brugere der har tid til at fordybe sig i de mange muligheder systemet stiller til rådighed, men giver dem hurtigt og enkelt mulighed for at passe deres arbejde på en tilfredsstillende måde for alle parter.
Jeg har i forbindelse med mit arbejde, ofte fået succesoplevelser i forbindelse med at man tidligt i processen med et IT-projekt har inddraget alle typer af brugere. Dette kan netop være en øjenåbner for hvordan man kan udforme et IT-system så alle kan benytte det. Senere i processen bliver det dog desværre tit til en prioritering af hvilket fokus der skal lægges ift. hvilken type bruger systemet skal være lavet for og det her kæden mange gange hopper af. Det er svært at lave IT-systemer der fokuserer på alle typer af brugere out-of-the-box, men man kan give brugerne mulighed for selv at træffe valg om hvad der er vigtigt for dem i stedet for at man som udvikler vælger for dem fra starten.
WCF-services hosted in a Windows service
Some time ago I ran into one of those problems with no obvious solution. I was on a project where I needed to use a WCF-service for a Silverlight solution. First I started out by making a service that was hosted on the IIS. That was working fine with a connection to a database, but the service was also going to open up some physical files in a specific folder on the Windows server where it was hosted. This could not be done as the service didn’t have access permissions to a folder outside of those under the service. So what to do then?
I searched for a solution to the problem and one that I found was to use Windows impersonation in the service. This “simulates” a user logged in on the server with the rights given to that user. For me this wasn’t an optimal solution for a number of reasons, first of all because it didn’t seem very secure. I quickly started to search for another way to cope with the problem.
The solution I came up with was this: I realized that as I had administrator rights to the server I could host my WCF-services in a Windows service and install it as such. In this way you can run the WCF-service outside of the IIS and run multiple services in the same Windows service as well. Another great thing about it is that if you install it right (as I will show below) you can get it to have access to the file system and it will run as a service under Windows. The example below shows you how the constructor in a Windows service can look like:
public Service()
{
InitializeComponent();
this.ServiceName = "Name of your Service";
this.EventLog.Log = "Application";
this.AutoLog = true;
this.CanHandlePowerEvent = true;
this.CanHandleSessionChangeEvent = true;
this.CanPauseAndContinue = true;
this.CanShutdown = true;
this.CanStop = true;
}
What happens here is that I provide the name the service has (in Windows this will be the name of the service) and if it should use the application log under Windows to log in. I also tell it that it should log automatically to the application event log when something happens with the service and I tell it that it’s okay to stop, shutdown and pause and continue among others.
The code example below shows the Main-method of the service. As with any other Windows application this has to be provided as the starting point of the application.
static void Main(string[] args)
{
try
{
ServiceBase.Run(new Service());
}
catch (Exception ex)
{
// Some logging or error handling here...
}
}
Here I make the service run. Remember the try-catch block because if something bad happens when the service is initialized this will not make everything crash. Also notice the inheritance from the ServiceBase-class. This is what makes our class a service and is needed when we’re going to install it later also and make it run. To make the service do something in certain situations before Windows service events is fired, when it starts, stops, continues, pauses or shuts down (if the server it resides under for instance shuts down), you can override the OnStart, OnStop, OnContinue, OnPause or OnShutdown methods respectively. Up until know I haven’t shown how I combine the Windows and WCF-services, but my next code examples shows just that. What you need first of all is to make your WCF-services start when the Windows service starts. This is done by overriding the OnStart method as mentioned above and then inside this hosting the WCF-services in some service hosts and open them. I learned that a good way to be able to control every service host with the WCF-service inside is to have it declared as variables inside the class. An example on this is provided below.
partial class Service : ServiceBase
{
public ServiceHost serviceHostFirstWcfService = null;
public ServiceHost serviceHostSecondWcfService = null;
....
}
In my OnStart method I do the following:
protected override void OnStart(string[] args)
{
try
{
if (this.serviceHostFirstWcfService != null)
this.serviceHostFirstWcfService.Close();
this.serviceHostFirstWcfService = new ServiceHost(typeof(FirstWcfService));
this.serviceHostFirstWcfService.Open();
if (this.serviceHostSecondWcfService != null)
this.serviceHostSecondWcfService.Close();
this.serviceHostSecondWcfService = new ServiceHost(typeof(SecondWcfService));
this.serviceHostSecondWcfService.Open();
}
catch (Exception ex)
{
// Some exception handling...
}
}
First of all for every WCF-service I find out whether it has been initialized before. If it has, I close it so I don’t get an exception thrown when I try to open a service that is already open. Afterwards I declare the service host given the type of WCF-service that it should open and open it. As with everything else I pack it all inside a try-catch block to prevent the Windows service from crashing.
Finally you need to provide the normal configuration settings for your WCF-services inside an App.config file in the Windows service class library. The App.config file can look like the one below:
There you have it! The service works fine like this and if you have provided the correct base address the service can be called from there (remember that what you provide as address in the baseAddress property as the full address for the service – so when you call it you don’t need the .svc prefix). Tip: You can provide multiple endpoints if you need multiple bindings.
The service worked great until I started to test it with the Silverlight application and realized that the service, of course, needed a clientaccesspolicy.xml file to have the right access rights for the WCF-service, but where do you put this when you don’t have your service on the IIS and you can’t just put the access policy file in the root directory? The solution for this is to understand how the Silverlight application asks for the clientaccesspolicy.xml file when calling the WCF-service. When a Silverlight application calls the service in another domain it automatically assumes that the clientaccesspolicy file is located in the root of the domain where the service resides (the base address provided in the configuration file without the service name). If it isn’t there it gives you the standard 404 error (”File not found” or something like that) when you try to call the service from the application. So if you can ”broadcast” a clientaccesspolicy.xml file on the root address this is the solution, but how to do that? The solution is to add a new WCF-service that, by using the HTTP GET-protocol stream the xml-file as a message. The interface for the new service should look like below. The UriTemplate-property of the WebGetAttribute tells what the URI should be. In our example it is the name of the clientaccesspolicy file.
[ServiceContract(Namespace = "http://YourService")]
public interface IClientAccessPolicyService
{
[OperationContract]
[WebGet(UriTemplate = "clientaccesspolicy.xml")]
Message ProvidePolicyFile();
}
Then, in the implementation of the service interface, you just open and read the clientaccesspolicy.xml file as a stream, load it into a StringReader, add this to a XmlReader, and make a new instance System.ServiceModel.Channels.Message, add your Xmlreader object to the message and return it.
public class ClientAccessPolicyService : IClientAccessPolicyService
{
public System.ServiceModel.Channels.Message ProvidePolicyFile()
{
try
{
string fileContent = string.Empty;
StreamReader fileStream = new StreamReader("C:\the\full\path\to\your\clientaccesspolicy.xml");
fileContent = fileStream.ReadToEnd();
fileStream.Close();
StringReader sr = new StringReader(fileContent);
XmlReader reader = XmlReader.Create(sr);
System.ServiceModel.Channels.Message result = Message.CreateMessage(MessageVersion.None, "", reader);
return result;
}
catch (Exception ex)
{
return null;
}
}
}
There you have it! When you now call your service from a Silverlight application, you will see that it gets the clientaccesspolicy.xml from the service.
After my service is done I can install it using a Windows installer or a basic console application. I chose to install it just by running a console application and you can see how this is done in this example: Installing a Windows service
Installing a Windows service
To install a service you need to use the ServiceProcessInstaller and the ServiceInstaller classes. These handle the installation of the Windows service as a process with the provided information. The Account-property on the ServiceProcessInstaller is used to tell what security context the service should run under when installed. I use LocalSystem here because I need my service to have access to my file system, but here you have to choose what best fits your situation of course. The username and password properties are used to tell what user the service should run as. Next for the properties of the ServiceInstaller you can specify how and when your service is started with the StartType-property and you can set the service name, display name, a description and then you need to assign the ServiceProcessInstaller instance as parent to the ServiceInstaller instance. Then you need to specify the context in which the service should be installed: the full path to the executable that holds the service as a commandline and a path to a log file if needed. Then you install it by calling the Install-method on the ServiceInstaller instance. Then, because we need to start up the service, we takes control of the installed service with the ServiceController-class and call the Start-method. That’s it! Your service is installed.
ServiceProcessInstaller processInstaller =
new ServiceProcessInstaller();
processInstaller.Account = ServiceAccount.LocalSystem;
processInstaller.Username = null;
processInstaller.Password = null;
ServiceInstaller serviceInstaller =
new ServiceInstaller();
serviceInstaller.StartType = ServiceStartMode.Automatic;
serviceInstaller.ServiceName = ServiceName;
serviceInstaller.DisplayName = ServiceDisplayName;
serviceInstaller.Description = ServiceDescription;
serviceInstaller.Parent = processInstaller;
String path = String.Format("/assemblypath={0}", ServiceExecutablePath);
String[] cmdline = { path };
String logFilePath = “C:\path\to\your\logfile.txt”
serviceInstaller.Context = new System.Configuration.Install.InstallContext(logFilePath, cmdline);
System.Collections.Specialized.ListDictionary state =
new System.Collections.Specialized.ListDictionary();
serviceInstaller.Install(state);
ServiceController serviceController =
new ServiceController(serviceInstaller.ServiceName);
serviceController.Start();
If you need to uninstall your service to update it for instance (that’s right, the service executable cannot be altered as long as the service is installed), you can use the below code to do just that. It should be quite straight forward. You just take control of the service as in the code above, figure out if the service is running, if it’s running then it has to be stopped before we can uninstall it, wait for it to stop and then gives the need context-information and then we uninstall by calling Uninstall on our ServiceInstaller object.
ServiceInstaller serviceInstaller =
new ServiceInstaller();
serviceInstaller.ServiceName = ServiceName;
ServiceController serviceController =
new ServiceController(serviceInstaller.ServiceName);
if ((serviceController.Status == ServiceControllerStatus.Running)
|| (serviceController.Status == ServiceControllerStatus.Paused))
{
serviceController.Stop();
serviceController.WaitForStatus(ServiceControllerStatus.Stopped, new TimeSpan(0, 0, 0, 15));
serviceController.Close();
}
String path = String.Format("/assemblypath={0}", ServiceExecutablePath);
String[] cmdline = { path };
String logFilePath = “C:\path\to\your\logfile.txt”
serviceInstaller.Context = new System.Configuration.Install.InstallContext(logFilePath, cmdline);
serviceInstaller.Uninstall(null);
Bootup the virtual haddisk
Fall last year (2009) I was attending a session with Scott Hanselman in Copenhagen, Denmark where he talked about the, back then, brand new features in ASP.NET MVC 2.0. After getting to the conference room at the hotel 15 minutes late, because he went to the wrong hotel (yes Copenhagen can be a large city to travel in
he started the session by telling about a new cool feature in Windows 7 where it’s possible to make a bootable virtual harddisk that you can attach in the Disk Manager and then boot up on just like if it was a normal installed OS.
This is cool because you don’t have to run it in the Virtual PC (as a x86 installation – why can’t MS just make the free version of VPC x64 based!?), you can distribute it on multiple PC’s and because it boots up normally it utilizes the hardware in the PC as the normal OS. In that way it becomes the golden mean between a rigid Virtual PC installation and the mess of having two OS-installations.
This I knew I had to try some time and yesterday I got the opportunity. I needed to run a Windows Server 2008 R2 Enterprise beside my Windows 7 installation to use some of the services in the server for a little project I’m working on. Here I will explain what I did and what I learned.
I have to tell you a very important thing I learned after my PC crashed and didn’t wont to boot up again after I thought I had got everything done exactly as in Scott’s blogposts: YOU NEED TO HAVE WINDOWS 7 ULTIMATE TO GET THIS TO WORK! Otherwise you will find yourself stuck, as I was, with a PC that gives you a strange boot error, a Logitech USB/Bluetooth keyboard that is not working outside of Windows and no PS2 keys anywhere near you so you can skip the error, because PS2 was “so outdated”. Now, when I make something like this from now on, I will have a PS2 keyboard no less than 1 meter away from me
The first thing to do is to make the bootable VHD (Virtual Hard Disk) from a Windows installation ISO. This blog from Scott will tell you what to do:
Step-By-Step: Turning a Windows 7 DVD or ISO into a Bootable VHD Virtual Machine
So I followed Scott’s description of how to make the VHD and I was able to get it done. Remember the /SKU:SERVERENTERPRISE setting in the cscript command if you want to have a enterprise server.
Then I moved on to this post from Scott telling about how to get it to appear when booting up the PC:
Less Virtual, More Machine – Windows 7 and the magic of Boot to VHD
When you have a VHD already you just need to go down to the paragraph with the title “Setting up your Windows Boot Menu to boot to an Existing VHD” and start from there. This is quit straight forward, but remember one thing when doing this: go to the Disk Manager and attach the VHD before doing any of the things in this post! Otherwise it won’t work.
After doing the bcdedit commands and veryfied that your VHD appears in the Windows Boot Manager (remember to be absolutely sure that everything is looking alright here before restarting your system!). The problem is that if the Windows Boot Manager finds that one of the entries is not correct it simply wont but anything, even the OS that is correct! When you boot up now you will have a new OS in the Windows Boot Manager menu that you can choose. The first time you startup the new OS you will have a Administrator user and it will prompt you to give it a password.
Then you are good to go!
The coolest thing about all this is that you have all your harddisks in the new system (the VHD will now be your C-drive) and you can move files from one harddisk to the other and even cooler, you can take the VHD, move it to another PC, add it to the Windows Boot Manager using the last post from Scott that I mentioned and you have a full functioning OS with all you need installed.
Have fun with it
New features in .NET 4.0
I just found this on MSDN where you can see what’s new in the whole .NET 4.0 framework:
What’s New in the .NET Framework 4
It’s still the RC framework so be patient with it
As it says above, it’s “subject to change”. But I think it’s as close to the final list as it can be.
One of the cool things that I have forgot I needed in C# that was in C++ is optional parameters in methods. This realy saves time.
Another cool thing is the Enum.TryParse. I have missed that so much as it has been a pain to work with enums. Now I can safely parse enums as I like.
The implementation of String.IsNullOrWhitespace method is also a very cool thing.
Pivot: New ways of working with data
You should really check this out I was introduced to today. It’s something new from MS Live Labs called Pivot:
www.getpivot.com
There’s a cool introduction on the website but you should especially check this video about it out:
Gary Flake: is Pivot a turning point for web exploration?
It’s some kind of new way to presenting and gathering large sets of data. The video here shows a WPF application and the engine I think is on his own computer, but I saw on the website that they have a Silverlight subset of it.
Definetly something I will try to work with some time. When I get my first hands on approach on this I will write a post about it.
Guest lecture from MS Denmark
Today I attended a guest lecture by Jasper Hedegaard Bojsen, CTO at Microsoft Denmark. The lecture focussed about the network effect, lock-in types, switching cost and the whole issue of choice of open document standards in the public authorities in Denmark.
I always try to attend these kinds of lectures and meetings because they are almost always quit interesting. And todays lecture was indeed interesting as it gave an insight in what Microsoft (especially in Denmark) thinks about the open office standards issue and what they are about to launch of new products and features.
Regarding the war of the standards (note the reference to “The war of the worlds”) Jasper was in the defense talking a bit about what their competitors does to influence the decision makers. He emphazised on the fact that there is not just two Office products in the world today. The choice of office product is not realy just black and white (OpenOffice and MS Office), but there is a lot of alternatives out there. He also said that the price for an MS Office pack was not that expensive as many believe it to be. Regarding ODF as a standard, Jasper told that OOXML was a more rich standard than ODF and that there is a lot of loss of information when converting a OOXML document to a ODF one as the ODF standard is much smaller. He also stressed the fact that MS had made it possible in their Office package to save to ODF but the people behind OpenOffice haven’t made it possible in their package to save to a OOXML document and he thought that this was quit strange because it had been possible to save to .doc for instance which he thought was much more difficult to develop.
Google Docs was mentioned and he talked about the problems about Google Docs not using a office standard for their service at all and their ability to change the appearence as they liked. He also talked about the update ratio of Google Docs (20 to 30 times updated last year) and why that is a problem for the users. To me it sounds like Microsoft sees Google as one of their biggest competitors right now.
The last part of the lecture, Jasper talked about the new products that Microsoft was launching earlier this year and with that the new webbased Office service. This looks and sounds quite interesting but few of my fellow students stressed the fact that this new webbased office service wasn’t that innovative and that Google in this field was the innovative one. In the light of how big an competitor Google is for MS, Jasper told that service would not work in Google Chrome as MS would not like to “help Google to a greater browser market share”. This is something that has been seen with the lack of support for Silverlight in Google Chrome.
A very interesting guest lecture from the CTO and I thaught that he had a lot of good points about the war on standards. I know that it’s just one side of the story but he was very honest in his views. I look forward to follow the war of the standards and what the outcome in Denmark and in the rest of the world will be. I don’t think that any office products just yet can compete with especially the usability in MS Office and I have both used Google Docs on work and OpenOffice for a period and they really lack alot from what MS has right now.
My first post ever
This is my very first blogpost ever so now I’m part of the blogosphere. Cool.
I made this blog so I could share some tips and tricks about development in .NET and other MS technologies with the rest of the world. It will also contain some information about IT in general.
